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Executive  Summary 


This  report  describes  an  Institute  for  Defense  Analyses  (IDA)  cost-benefit  analysis 
of  a  project  to  improve  the  Navy’s  Integrated  Logistics  Support  (ILS).  The  project 
involves  integrating,  or  “bridging,”  the  management  of  technical  information  for  pro¬ 
ducing  Navy  technical  manuals  and  training  courses.  Integration  would  lower  the  cost  of 
producing  manuals  and  courses  and  would  increase  shipboard  readiness  by  having  the 
appropriate  logistics  support  on  hand  when  new  systems  and  equipment  upgrades  are 
fielded.  The  Bridge  project  achieved  the  integration  by  designing  new  software  and  pro¬ 
posing  new  technical  and  business  procedures  for  managing  the  technical  information. 

The  Bridge  project  was  funded  by  the  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics  (OUSD(AT&L)).  It  was  part  of  the  Reduction  in 
Total  Ownership  Cost  (RTOC)  program  conducted  by  the  Office  of  the  Secretary  of 
Defense  (OSD).  The  initial  beneficiary  of  the  funding  is  the  Littoral  Combat  Ship  (LCS) 
Mission  Modules  Program  (PMS  420),  which  is  integrating  the  mission  modules  into  the 
LCS. 

The  costs  of  the  Bridge  project  are  those  for  investment  and  implementation. 
Investment  is  the  personnel  and  related  expenses  of  the  project.  Implementation  is  the 
expenses  of  training  technical  writers  and  course  developers  in  using  the  Bridge,  plus  the 
license  and  user  fees  to  cover  the  additional  costs  of  maintaining  the  networks  and  the 
repositories  for  processing  and  storing  the  technical  information. 

The  study  conducts  separate  cost-benefit  analyses  for  two  perspectives.  The  OSD 
perspective  recognizes  OSD’s  broad  interest  in  seeing  whether  the  new  software  and 
technical  and  business  processes  that  constitute  the  Bridge  will  lead  to  net  cost  savings — 
benefits  exceeding  costs — if  implemented  by  the  Navy  and  other  Services  as  a  whole. 
Analysis  of  this  perspective  is  therefore  conducted  for  an  “aggregate”  sample:  the  Navy’s 
yearly  production  of  all  Hull,  Mechanical,  and  Electrical  (HM&E)  technical  manuals 
produced  by  the  Naval  Ship  System  Engineering  Station  (NAVSSES)  in  Philadelphia, 
and  all  Computer-Based  Training  (CBT)  courses  delivered  by  Navy  eLearning  (NeL),  a 
part  of  the  Naval  Education  and  Training  Command  (NETC). 

The  second  perspective  reflects  the  focus  of  Program  Offices  on  their  individual 
systems  of  interest.  The  analysis  for  the  Program  Office  perspective  concentrated  exclu¬ 
sively  on  the  benefit  side — whether  the  Bridge  would  save  money  in  producing  future 
technical  manuals  and  training  courses.  It  focused  on  the  benefits  of  a  “single-system” 
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sample:  the  AN/AQS-20A  mine-hunting  sonar  for  the  LCS.  To  expect  that  these  savings 
would  cover  the  full  investment  and  implementation  costs  of  the  Bridge  would  be 
unreasonable. 

The  aggregate  analysis  finds  that  the  Bridge  would  achieve  net  benefits  of 
$78.1  million  in  10-year  costs:  a  savings  of  $86.8  million  in  producing  future  HM&E 
manuals  and  NeL-delivered  courses  less  $8.7  million  in  investment  and  implementation 
costs.  The  single-system  analysis  finds  that  the  Bridge  would  produce  substantial  savings 
of  almost  $306,  000  over  10  years. 

These  results  are  uncertain  because  of  the  newness  of  the  Bridge.  Although  much  of 
the  analysis  is  based  on  historical  data,  some  of  the  inputs  are  projections  of  the  new 
Bridge’s  productivity.  A  sensitivity  analysis  of  the  five  most  uncertain  inputs  shows  a 
range  of  10-year  net  benefits  for  the  aggregate  analysis  varying  from  $32  million  to 
$120  million.  These  benefits  would  be  much  greater  if  the  Bridge  were  applied  to  the 
technical  manuals  and  training  courses  of  the  entire  Navy  and  other  Services.  This  intri¬ 
guing  innovation  deserves  further  study. 
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1.  Introduction 


This  report  describes  an  Institute  for  Defense  Analyses  (IDA)  cost-benefit  analysis 
of  a  project  to  improve  the  Navy’s  Integrated  Logistics  Support  (ILS).  The  project 
involves  integrating  (“bridging”)  the  management  of  technical  information  for  producing 
Navy  technical  manuals  and  training  courses  to  support  fielded  systems  and  equipment 
upgrades.  The  principal  benefits  analyzed  in  this  study  would  be  lower  costs  of  producing 
future  manuals  and  courses.  Also  included  is  a  parametric  analysis  of  the  improvement  in 
shipboard  readiness  that  results  from  supporting  newly  fielded  systems  and  equipment 
upgrades  with  the  most  reliable  and  timely  information. 

The  project  was  conducted  for  the  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  (OUSD(AT&L)).  It  was  part  of  the  Reduction  in 
Total  Ownership  Cost  (RTOC)  program  conducted  by  the  Office  of  the  Secretary  of 
Defense  (OSD).  The  initial  beneficiary  of  the  funding  is  the  Littoral  Combat  Ship  (LCS) 
Mission  Modules  Program  (PMS  420). 

The  integration  results  from  new  software  and  technical  and  business  processes  the 
project  developed  to  integrate  the  production  of  technical  manuals  and  training  courses. 
The  new  methods  would  be  used  in  conjunction  with  the  S1000D  technical  data  specifi¬ 
cation  and  the  Sharable  Content  Object  Reference  Model  (SCORM). 1 

A.  Terminology 

The  meaning  of  “Bridge”  in  this  project  has  changed.  It  originated  in  the  early  plan¬ 
ning  stage  conducted  during  September-December  2008.  At  that  time,  the  project’s  focus 
was  on  identifying  why  Naval  technical  training  content  is  often  out  of  date  and  not  fac¬ 
tored  into  system  life-cycle  management. 

An  analysis  of  Navy  management  and  production  practices  revealed  that  tools  for 
processing  learning  content  cannot  easily  access  Common  Source  Data  Bases  (CSDBs) 
that  contain  related  technical  data.  The  project’s  focus  therefore  shifted  to  developing  an 
Application  Programming  Interface  (API)  to  improve  this  access.  The  API  became 
known  as  the  “API  Bridge. 


SCORM  is  a  group  of  specifications  for  standardizing  the  packaging  of  training  content — not  the  content 
itself — so  it  can  be  delivered  by  any  Learning  Management  System  (LMS).  An  LMS  is  a  Web-based 
e-learning  system. 
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However,  as  the  project  matured,  it  became  apparent  that  using  the  API  Bridge 
required  additional  innovation:  a  package  of  software  tools  and  technical  and  business 
processes  to  foster  interoperability  by  integrating  technical  data  and  learning  content 
throughout  system  life  cycles.  This  entire  package  of  improvements,  now  referred  to  as 
the  “Bridge”  project,  is  the  subject  of  this  cost-benefit  analysis. 

B.  Outline  of  the  Report 

This  report  is  organized  around  the  following  topics: 

•  How  technical  manuals  and  training  courses  are  currently  produced  by  non-inte- 
grated  processes 

•  The  Bridge  project’s  proposals  to  integrate  these  processes 

•  The  analytical  approach  to  estimating  the  costs  and  benefits  of  the  proposals 

•  The  quantitative  analysis  of  costs  and  benefits 

The  first  three  topics  are  addressed  in  the  remainder  of  this  chapter.  Section  1  .C  dis¬ 
cusses  the  shortfalls  of  not  integrating  the  production  of  technical  manuals  and  training 
courses,  Section  l.D  describes  the  Bridge  Project’s  proposals  to  relieve  the  shortfalls,  and 
Section  l.E  outlines  the  cost-benefit  analysis — the  overall  analytical  approach  and  the 
particular  costs  and  benefits  to  be  evaluated.  The  chapter  ends  with  caveats  (Section  l.F). 

The  fourth  topic,  the  quantitative  analysis  of  costs  and  benefits,  is  addressed  in  suc¬ 
ceeding  chapters:  the  investment  and  implementation  costs  of  the  Bridge  (Chapter  2),  the 
benefits  measured  by  the  reduction  in  costs  of  producing  future  technical  manuals  and 
training  courses  using  the  Bridge  (Chapter  3),  and  the  benefits  through  improvements  in 
Fleet  readiness  (Chapter  4).  The  summary  (Chapter  5)  presents  a  sensitivity  analysis  of 
the  major  uncertainties  and  briefly  describes  the  need  for  additional  work. 

Several  appendixes  augment  material  referred  to  in  the  body  of  the  report: 

•  Appendix  A.  Detailed  Staff-Hour  Estimates  for  Producing  a  Nominal  500-Page 
Technical  Manual  Under  Current  and  Bridge  Processes 

•  Appendix  B.  Detailed  Staff-Hour  Estimates  for  Producing  a  Nominal  One-Con¬ 
tent-Hour  Training  Course  Under  Current  and  Bridge  Processes 

•  Appendix  C.  Timeliness  of  Technical  Manuals  and  Training  Courses 

•  Appendix  D.  Navy  Learning  Centers 

•  Appendix  E.  Content  Hours  and  Computer-Based  Training  (CBT)  Courses  Deli¬ 
vered  by  Navy  eLearning  (NeL) 
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C.  Current  Non-Integrated  Production  of  Navy  Technical  Manuals 
and  Training  Courses 

A  major  goal  of  ILS  is  that  crucial  support  resources,  such  as  technical  manuals  and 
training  courses,  are  in  place  when  new  systems  and  equipment  are  deployed.-  Although 
the  goal  is  clear,  its  attainment  often  falls  short  for  two  major  reasons: 

•  Technical  training  and  learning  and  performance  requirements  are  not  consis¬ 
tently  factored  into  the  early  system  acquisition  documents  (e.g.,  milestone  doc¬ 
uments  and  Acquisition  Logistics  Support  Plans  (ALSPs)). 

•  The  methods  for  producing  technical  data  and  learning  content  fail  to  make 
maximum  use  of  interoperability  to  achieve  integration.  This  lack  of  integration 
causes  two  major  problems:  technical  manuals  and  training  courses  cost  more  to 
produce  because  of  duplication  of  effort,  and  disparities  may  appear  in  the  con¬ 
tent  and  timeliness  of  the  information  they  present. 

The  second  reason  is  the  subject  of  the  present  analysis.  Here  are  the  particular  fea¬ 
tures  of  non-integrated  production  that  lead  to  unnecessary  cost: 

•  Production  process.  Technical  manuals  and  training  courses  are  generally  pro¬ 
duced  by  separate  processes.  Course  developers  use  technical  manuals  as  inputs, 
but  they  also  obtain  their  own  contractor  data  and  job  task  analyses  and  then 
process  the  data  using  separate  techniques. 

•  Authoring  tools.  Training  course  developers  use  authoring  tools  (editors)  that 
do  not  have  easy  access  to  the  CSDBs  where  technical  data  are  stored. 

•  Storage  format.  Technical  and  training  data  are  stored  in  different  fonnats  that 
are  not  designed  for  easy  integration  and  data  reuse. 

•  Storage  repositories.  Technical  and  learning  data  are  usually  stored  in  different 
and  disconnected  repositories. 

•  System  and  equipment  upgrading.  Although  fielding  of  new  systems  is  typi¬ 
cally  followed  by  periodic  equipment  upgrades  described  by  such  mechanisms 
as  Engineering  Change  Proposals  (ECPs),  we  lack  common  or  automated  identi¬ 
fication  systems  (metadata)  or  technical  and  business  processes  for  linking  ECPs 
to  the  specific  technical  and  training  content  that  might  require  modification. 


Defense  Acquisition  University  course  in  Fundamentals  of  Systems  Acquisition  Management,  Lesson  19 
(Acquisition  Logistics:  Fundamentals)  and  Lesson  20  (Acquisition  Logistics:  Supportability  Planning). 
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This  lack  of  integration  can  lower  Fleet  readiness  through  several  mechanisms: 

•  Disparities  in  information.  Technical  manuals  and  training  courses  may  not 
deliver  the  same  information. 

•  Information  lags  following  the  fielding  of  new  systems.  Delayed  production 
and  distribution  of  manuals  and  courses  can  create  the  possibility  that  the  sup¬ 
porting  technical  information  might  lag  the  fielding  of  new  systems.  (Courses 
may  always  lag  the  manuals  to  which  they  turn  for  information,  but  these 
courses  and  manuals  must  be  available  for  newly  fielded  systems  to  ensure 
readiness.) 

•  Information  lags  following  equipment  updates  via  ECPs  and  similar 
mechanisms.  Delayed  production  might  also  create  lags  in  updating  manuals 
and  courses  in  response  to  equipment  upgrades  through  ECPs. 

Over  time,  experienced  maintenance  technicians  develop  a  general  understanding  of 
how  equipment  works  and  may  not  always  need  up-to-date  manuals  or  courses  for  a  par¬ 
ticular  repair.  Nevertheless,  providing  ships  the  up-to-date  documents  is  the  only  way  to 
ensure  that  the  correct  information  is  always  available  when  needed. 

D.  The  Bridge  Project  To  Integrate  the  Production  of  Technical 
Manuals  and  Training  Courses 

The  Bridge  project  engaged  in  three  broad  efforts  to  relieve  the  problems  of  high 
cost  and  low  readiness: 

1 .  Identification  of  new  technical  and  business  processes  for  integrating  the  pro¬ 
duction  of  technical  manuals  and  training  courses 

a.  The  textual  data  for  all  manuals  from  Original  Equipment  Manufacturers 
(OEMs)  would  be  written  by  the  OEMs  or  the  Navy  in  the  same  digital  for¬ 
mat:  the  S1000D  Issue  4  Data  Module  (DM)  technical  data  specification. 
S1000D  is  an  international  digital  specification  for  information  used  in  the 
procurement  and  production  of  technical  publications.  It  is  a  Standard  Gene¬ 
ralized  Markup  Language/Extensible  Markup  Language  (SGML/XML) 
standard.  Non-textual  material  (e.g.,  illustrations)  would  be  called  up  by 
S1000D  statements.  DMs  contain  data  arranged  by  product  components  and 
subcomponents.  One  DM,  for  example,  might  contain  all  of  the  power  sup¬ 
plies  for  DD-963  fire-control  systems.  Data  organized  in  this  fashion  would 
make  reuse  easier. 

b.  All  DMs  would  be  stored  in  CSDBs.  The  Navy  has  already  begun  storing 
data  for  many  of  its  systems — including  systems  for  submarines  and  the 
Aegis  guided-missile  cruisers — into  a  single  Navy  CSDB. 
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c.  Course  developers  would  use  a  common  interface  (i.e.,  the  API)  to  read  the 
technical  data  in  the  S1000D  DMs  stored  in  CSDBs.  Using  these  data  and 
information  from  job  performance  analyses  and  OEMs,  course  developers 
would  structure  SCORM-conformant  learning  content  that  could  be  used  in 
any  Learning  Management  System  (LMS). 

d.  All  learning  content  would  be  structured  in  S1000D  Learning  Data  Modules 
(LDMs)  for  storage  back  into  the  CSDBs.  S1000D  Issue  4,  released  in  2009, 
can  accommodate  LDMs.  As  with  technical  data,  non-textual  learning  con¬ 
tent  (e.g.,  illustrations)  would  be  called  up  by  S1000D  statements. 

e.  All  DMs  and  LDMs  would  be  described  by  the  S 1000D  file-naming  conven¬ 
tion,  the  Data  Module  Code  (DMC).  The  DMC,  a  37-digit  number  at 
present,  represents  physical  and  functional  information  for  the  systems  con¬ 
tained  in  the  various  DMs  and  LDMs. 

f.  A  software  application  (the  Web  Service)  would  be  used  to  identify  the 
DMs  and  LDMs  that  must  be  reviewed  for  possible  modification  in  response 
to  equipment  upgrades  specified  by  ECPs. 

2.  Development  of  the  API  and  Web  Service 

3.  Demonstration  of  the  feasibility  and  attractiveness  of  applying  the  new  technical 

and  business  processes  to  the  production  of  technical  manuals  and  training 

courses 

a.  Converting  the  PowerPoint-based,  instructor-led  training  for  the  AN/AQS- 
20A  into  S1000D  LDMs. 

b.  Converting  an  AN/AQS-20A  technical  manual  from  a  normal  “page¬ 
turning”  document  composed  in  Microsoft  Word  and  Microsoft  PowerPoint 
into  S1000D  Issue  4  DMs. 

c.  Conducting  the  present  cost  and  benefit  analysis  of  the  Bridge. 

All  of  these  tasks  were  completed  in  preliminary  draft  during  the  first  year  of  the 
project.  Several  tasks  will  be  performed  in  the  second  year  of  the  project  to  fully  imple¬ 
ment  the  new  products  into  the  Navy  infrastructure.  These  tasks  include  reviewing  the 
initial  programming  of  the  API  and  Web  Service  and  forming  the  S1000D  AN/AQS-20A 
technical  manuals  and  training  courses  into  SCORM  packages  for  input  into  the  Navy 
CSDB. 


The  API  would  allow  course  developers  who  use  different  authoring  tools  to  exchange  data  easily  with 
any  CSDB. 
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E.  Outline  of  the  Cost-Benefit  Analysis 


1.  General  Description 

The  analysis  will  estimate  the  following  costs  and  benefits  of  the  project: 

•  Costs 

-  Investment 

-  Implementation 

•  Benefits 

-  Future  cost  savings 

-  Contributions  to  Fleet  readiness 

The  costs  and  benefits  are  estimated  over  10  years  and  discounted  to  FY  2010  present 
values  using  the  annual  2.4%  discount  rate  mandated  for  10-year  projects  by  the  Office  of 
Management  and  Budget  (OMB)  Circular  A-94,  Guidelines  and  Discount  Rates  for  Ben¬ 
efit-Cost  Analysis  of  Federal  Programs  (Appendix  C  of  the  Circular,  January  2009 
update). 

The  investment  costs  are  the  project’s  expenses  for  completing  the  second  year  of 
the  project.  The  first  year  was  largely  completed  when  the  cost-benefit  analysis  was 
begun.  The  implementation  costs  are  the  estimated  future  expenses  for  training  Navy  per¬ 
sonnel  in  using  the  Bridge  and  for  the  site/program  licenses  and  user  fees  to  cover  the 
costs  of  maintaining  and  upgrading  the  networks  and  CSDBs. 

The  first  benefit  is  the  cost  reduction  achieved  by  using  the  Bridge  to  produce  future 
technical  manuals  and  training  courses.  These  savings  were  calculated  in  three  steps: 

•  Estimating  the  reduction  in  staff  hours  for  producing  a  single  nominal  technical 
manual  and  a  single  nominal  training  course, 

•  Applying  pay  rates  to  convert  the  staff  hour  savings  to  cost  savings,  and 

•  Scaling  up  the  savings  for  the  nominal  manual  and  course  to  the  estimated 
10-year  production  of  the  two  samples  of  Navy  technical  manuals  and  training 
courses  described  in  the  next  subsection. 

The  second  benefit  is  the  improvement  in  Navy  readiness  that  will  occur  by  pro¬ 
viding  ships  newly  fielded  systems  and  equipment  upgrades  only  when  the  appropriate 
logistics  support  is  available  (see  Appendix  C  of  this  paper).  In  this  case,  we  lacked  cur¬ 
rent  Fleet  data  and  performed  a  parametric  analysis  instead. 
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2.  Cost  Savings 

We  analyzed  the  cost  savings  for  two  samples  based  on  the  different  perspectives  of 
OSD  and  the  PMS  420  Program  Office.  PMS  420 ’s  task  is  to  integrate  the  mission  mod¬ 
ules  into  the  LCS. 

The  OSD  perspective,  reflected  in  its  RTOC  program,  is  to  achieve  net  savings 
(benefits  exceeding  cost)  for  all  defense  systems.  We  analyzed  an  aggregate  sample  for 
this  case:  all  Hull,  Mechanical,  and  Electrical  (HM&E)  technical  manuals,  which  are  pro¬ 
duced  by  the  Naval  Ship  Systems  Engineering  Station  (NAVSSES)  in  Philadelphia,  and 
all  CBT  courses  delivered  by  NeL,  which  is  a  part  of  the  Naval  Education  and  Training 
Command  (NETC).  The  question  for  this  aggregate  analysis  is  whether  this  sample’s 
savings  cover  the  investment  and  implementation  costs.  The  aggregate  sample  is  not 
complete,  since  it  excludes  the  technical  manuals  for  electronic  and  ordnance  components 
and  the  Navy  CBT  courses  other  than  those  delivered  by  NeL.  However,  the  sample  is 
large  enough  to  indicate  whether  the  benefits  will  cover  the  costs  if  the  Bridge  is  imple¬ 
mented  over  a  large  force. 

The  Program  Office  perspective,  by  comparison,  focuses  on  a  single  system,  the 
AN/AQS-20A  underwater  sonar  mine  detector  that  is  towed  by  the  MH-60S  helicopter  as 
part  of  the  LCS  mine-warfare  mission  package.  Interest  here  is  only  on  the  benefit  side: 
Would  the  Bridge  save  money  in  producing  future  technical  manuals  and  training  courses 
without  regard  for  the  investment  and  implementation  costs?  Determining  whether  the 
savings  for  a  single  system  would  cover  the  full  investment  costs  of  the  Bridge  is  not  of 
interest. 

3.  Improvements  in  Readiness 

Lowering  cost  is  not  the  only  benefit  of  the  Bridge.  Improvements  in  Fleet  readiness 
are  also  possible.  A  recent  Navy  policy,  described  in  Appendix  C,  dictates  that  new  sys¬ 
tems  and  equipment  upgrades  should  not  be  installed  on  ships  during  yard  availabilities 
unless  the  support  logistics — up-to-date  technical  manuals  and  training  courses — are 
available.  The  Bridge  supports  this  policy,  since  integrating  the  production  of  technical 
manuals  and  training  courses  is  expected  to  improve  the  consistency  of  technical  infor¬ 
mation  and  its  timely  delivery  to  the  Fleet.  We  carried  out  a  parametric  analysis  for 
insight  into  whether  the  benefits  might  be  significant.  This  analysis  used  cost  data  for  the 
new  Zumwalt  DDG-1000  class  destroyers. 

F.  Caveats 

The  Bridge  involves  the  development  of  new  software  and  new  technical  and  busi¬ 
ness  procedures  for  managing  technical  data  and  related  training  content.  Many  of  the 
inputs  for  this  analysis  are  based  on  historical  data: 
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•  NAVSSES  data  on  the  production  of  HM&E  technical  manuals, 

•  NETC  data  on  the  production  of  training  courses  delivered  by  NeL, 

•  The  pay  rates  of  people  who  produce  technical  manuals  and  training  courses  for 
the  AN/AQS-20A  system  at  the  Naval  Surface  Warfare  Center  (NSWC)  in 
Panama  City,  Florida,  and 

•  The  site  program  licenses  and  user  fees  for  supporting  the  CSDBs  and  networks. 

The  cost  savings,  however,  are  also  based  on  the  projections  of  the  final  costs  and 
the  future  capability  of  the  Bridge  at  the  halfway  point  in  the  Bridge  project.  This  study  is 
the  first  time  that  the  cost  savings  of  integrated  production  have  been  estimated,  and  the 
training  course  savings  are  derived  from  experience  in  only  1  of  the  14  Learning  Centers 
listed  in  Appendix  D.  Chapter  5  includes  a  sensitivity  analysis  of  five  uncertain  inputs 
that  significantly  affect  the  quantitative  results.  The  second  year  of  the  project  will  be 
devoted,  at  least  in  part,  to  efforts  to  refine  these  inputs. 


1-8 


2.  Investment  and  Implementation  Costs 


A.  Investment  Costs 

Table  2-1  shows  the  full  investment  costs  for  the  Bridge  project,  which  are  the 
expenses  for  conducting  the  project  during  the  2  years  between  February  2009  and  April 
2011.  The  first  year  expenses  are  sunk  costs  since  they  were  largely  spent  when  the  cost- 
benefit  analysis  was  begun  and  are  therefore  excluded  from  the  analysis.  Because  they 
are  based  on  historical  data,  however,  they  are  used  to  estimate  the  costs  for  the  second 
year,  whose  tasks  are  similar  to  those  of  the  first  year. 


Table  2-1.  Bridge  Project  Investment  Costs 
(Full  Cost  to  the  Government) 


Year 

Cost 
(FY10  $) 

Year  1,  February  2009-February  2010  Note 

1.77  million 

Year  2,  February  201 0— April  201 1 

1.79  million 

Total 

3.56  million 

Note  for  Table  2-1:  These  are  sunk  costs  not  included  in  the  comparisons  with  the  future  benefits  of  the 
Bridge. 

The  investment  costs  include  the  fully  burdened  salaries  of  the  contractors  and  gov¬ 
ernment  personnel  who  worked  on  the  study,  plus  their  travel,  Other  Direct  Charges,  and 
the  costs  of  administering  the  contract.  The  contractor  costs  are  the  figures  from  the  Navy 
contract  for  the  project  and  are  thus  fully  burdened  without  modification.  (No  additional 
administrative  costs  were  incurred  since  the  contractors  generally  work  in  their  firms’ 
offices.)  The  costs  of  the  government  personnel  were  obtained  by  multiplying  their  staff 
hours  by  their  General  Schedule  (GS)  base  pay  rates,  increased  by  the  burdening  allow¬ 
ances  shown  in  Table  2-2.  The  2%  2010  government  pay  raise  the  president  signed  in 
December  2009  was  added  to  bring  costs  to  2010  dollars. 

B.  Implementation  Costs 

Table  2-3  shows  the  implementation  costs  for  the  aggregate  sample:  $2.86  million 
for  the  HM&E  technical  manuals  and  $4.02  million  for  the  NeL  training  courses.  The 
implementation  costs  consist  of  three  components:  (1)  Trainee  Personnel,  the  salary  of 
personnel  enrolled  in  a  postulated  2-week  course  to  train  technical  writers  and  course 
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Table  2-2.  Burdening  Allowances 


Locality  differential  for  DC/MD/VA/WV/PA 

23.10% 

Fringe  benefitsNote  1 

35% 

OverheadNote2 

40% 

G&ANote  3 

10% 

2010  pay  raise 

2% 

Total  allowanceNote4 

161% 

Note  1  for  Table  2-2:  Medical,  retirement,  and  so  forth. 

Note  2  for  Table  2-2:  Secretarial  support  staff,  building  rental,  utilities,  and  so  forth. 

Note  3  for  Table  2-2:  Management  and  departments  (human  resources,  accounting,  computer,  and  so 
forth). 

Note  4  for  Table  2-2:  The  161%  is  the  resultant  of  the  previous  allowances:  (1  +  23. 1  %)*(1  +  35%)*(1  + 
40%)*(1  +  10%)*(1  +  2%)-  1  =  1.61  (161%). 


Table  2-3.  Implementation  Costs  for  the  Aggregate  Sample 


Cases 

Technical 

Manuals 

Training 

Courses 

Trainee  Personnel 

Number  of  people  who  require  training 

70 

170 

Bridge  training  session  (hours) 

80 

80 

Hourly  contractor  pay  rate 

$65.00 

$65.00 

One-time  cost 

$364,000 

$883,830 

Occurrences  in  10  years 

4 

4 

10-year  cost,  present  value  (FY10$) 

$809,947 

$1,966,638 

Trainee  Course 

Staff  hours  to  produce  a  one-content-hour  course 

360 

360 

Number  of  content  hours  for  training  session 

10 

10 

Average  hourly  pay 

$65.00 

$65.00 

One-time  cost 

$234,000 

$234,000 

Occurrences  in  10  years 

4 

4 

10-year  cost,  present  value  (FY10  $) 

$520,680 

$520,680 

Site/Program  Licenses  and  User  Fees 

Cost  per  site/program 

$25,000 

$25,000 

Number  of  sites/programs 

6 

6 

Cost  per  user 

$1,200 

$1,200 

Number  of  users 

20 

20 

One-time  cost 

$147,800 

$147,800 

Occurrences  in  10  years 

10 

10 

10-year  cost,  present  value  (FY10  $) 

$1,300,265 

$1,300,265 

Total 

$2,861,386 

$4,018,076 
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developers  in  using  the  Bridge,  (2)  Trainee  Course,  construction  of  a  trainee  CBT 
course  for  use  in  the  2-week  training  course,  and  (3)  Site/  Program  Licenses  and  User 
Fees  to  cover  the  additional  costs  of  maintaining  and  upgrading  networks  and  CSDBs.  A 
NAVSSES  official  believes  that  this  training  would  enable  the  Navy  to  obtain  Bridge- 
capable  technical  writers  at  the  same  pay  rates  that  it  is  now  paying  for  SGML  writers. 

We  lacked  data  to  estimate  the  implementation  costs  for  the  AN/AQS-20A  case. 
The  workloads  for  this  case  (number  of  pages  of  technical  manuals  and  number  of  con¬ 
tent  hours  of  training  courses)  are  approximately  1%  of  those  for  the  aggregate  case,  so 
the  implementation  costs  would  also  be  much  lower. 


1.  Trainee  Personnel 

The  personnel  cost  for  an  individual  training  session  is  the  product  of  (1)  the 
instructor  and  the  number  of  people  who  require  training,  (2)  the  number  of  hours  in  the 
estimated  2-week  training  period,  and  (3)  the  contractor  average  hourly  pay  rate  calcu¬ 
lated  in  Table  2-4.  The  10-year  present  value  assumes  that  the  training  session  is  held 
four  times  during  the  10-year  period  (every  3  years,  in  years  1,  4,  7,  and  10)  because  of 
staff  changes  through  retirement  and  new  hires  and  because  of  assumed  improvements  in 
techniques  and  technology.  Since  these  changes  accumulate  over  time,  we  assumed  that 
the  follow-on  training  sessions  would  become  more  important  over  time  and  multiplied 
the  one-time  cost  by  30%,  50%,  and  70%  in  years  4,  7,  and  10,  respectively.  Thus,  the 
present  value  of  a  one-time  cost  of  $1  calculated  using  a  2.4%  discount  rate  is  calculated 
as  follows: 


Present  value  = 


$i 


|  $1x30%  |  $1x50%  |  $1x70%  _^2  23  (2  1) 


(1  +  2.4%)!  (1  +  2.4%)4  (1  +  2.4%) 7  (1  +  2.4%)10 


Table  2-4.  Hourly  Pay  Rates 


Use 

Pay  Rate 

Training  courses,  contractors  in  some  past  studies 

$64.89 

Training  courses,  government  in  some  past  studies 

$61.35 

AN/AQS-20A  for  technical  manuals  and  training  courses 

$65.00 

Used  in  this  study 

$65.00 

The  number  of  technical  writers  who  require  training  for  the  aggregate  sample  (70) 
is  a  NAVSSES  estimate  of  the  number  of  technical  writers  who  are  not  already  fluent  in 
S1000D,  DMs,  and  CSDBs.  The  number  of  course  developers  who  require  Bridge 
training  (170)  is  derived  in  Table  2-5.  The  total  number  of  staff  (760)  was  calculated  by 
multiplying  the  number  of  content  hours  delivered  by  NeL  in  2009  (3,292  hours,  see 
Table  2-6  and  refer  to  Appendix  E  for  more  details)  by  the  number  of  staff  hours 
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Table  2-5.  Number  of  Course  Developers  Requiring  Bridge  Training 


Number  of  NeL  content  hours  produced  in  2009 

3,292 

Number  of  staff  hours  to  produce  one-content-hour  training  course 

425 

Total  number  of  staff  hours  per  year 

1,399,100 

Number  of  staff  hours  per  staff  member  per  year 

1,840 

Total  number  of  staff 

760 

Percentage  of  staff  requiring  training 

22.4% 

Total  staff  requiring  training 

170 

Table  2-6.  Production  of  CBT  Delivered  by  NeL 


Year 

Total  Content  Hours 

2004 

1,637 

2005 

2,196 

2006 

2,207 

2007 

4,124 

2008 

3,322 

2009 

3,292 

to  produce  a  1 -content-hour  training  course  (425  hours,  derived  in  Appendix  B)  and 
dividing  the  resulting  total  staff  hours  per  year  by  1,840,  the  number  of  working  hours 
per  year,  assuming  2  weeks  for  sick  leave,  2  weeks  for  annual  leave,  and  10  federal  holi¬ 
days.  The  resulting  760  staff  members  are  multiplied  by  22.4%,  the  percentage  of  total 
staff  hours  (calculated  in  Table  2-7)  contributed  by  people  with  the  professional  skills 
assumed  to  require  Bridge  training. 

The  3,292  content  hours  that  NeL  delivered  in  2009  were  produced  by  various  Navy 
agencies.  We  used  the  estimate  for  2009  from  Table  2-6  without  extrapolation  because 
although  the  trend  is  up,  it  is  not  consistent. 

The  $65.00  hourly  pay  rate  from  Table  2-4  is  the  average  fully  burdened  rate  paid 
for  contractors  involved  in  producing  technical  manuals  and  training  courses  for  the 
AN/AQS-20A  system  at  NSWC  in  Panama  City,  Florida.  As  the  other  rates  in  the  table 
show,  it  is  close  to  the  historical  pay  rates  paid  for  some  recent  training  courses.  We  will 
use  this  pay  rate  in  the  following  calculations  but  consider  the  sensitivity  of  the  final 
results  to  this  number  in  Chapter  5. 

2.  Trainee  Course 

The  cost  of  computer-based  software  for  the  training  session  is  the  product  of  three 
factors:  (1)  the  number  of  staff  hours  to  produce  a  1 -content-hour  training  course 
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Table  2-7.  Professional  Skills  Involved  in  Producing  Training  Courses 


Staff  Hours  for  Developing  a 

1 -Content-Hour  Course 

Professional  Skill 

Total 

Requiring  Training 

Training  Director 

2 

Rate  Lead  (Subject  Matter  Expert  (SME)) 

90 

Learning  Standards  Officer 

29 

29 

Content  Sponsor 

4 

Course  Supervisor 

8 

Contracting  Officer 

1 

Instructional  Systems  Designer 

43 

Graphics 

168 

Animation 

14 

Programmer 

31 

31 

Editor 

29 

29 

Lead 

6 

6 

Total 

425 

95  (22.4%) 

derived  in  Appendix  B,  (2)  the  assumed  number  of  content  hours  per  training  session,  and 
(3)  the  average  fully  burdened  hourly  pay  of  $65.00  from  Table  2-4.  The  10-year  cost  is 
calculated  as  before. 


3.  Site/Program  Licenses  and  User  Fees 

Weapons  and  support  programs  are  currently  charged  site/program  licenses  and  user 
fees  under  a  Navy-vendor  CSDB  license  agreement.  (NAVSSES  licenses  are  charged  by 
the  site  and  by  the  program.)  The  license  fees  are  paid  to  NAVSSES  to  cover  the  costs  of 
maintaining  the  infrastructure,  and  the  user  fees  are  paid  to  vendors  to  cover  the  costs  of 
maintaining  the  networks  and  CSDBs.  The  annual  costs  are  calculated  as  follows: 

Annual  cost  for  licenses  and  fees  =  $25,000  x  (number  of  sites/programs) 

+  $1,200  x  (number  of  users).  (2.2) 


The  10-year  present  value  is  calculated  as  in  Eq.  2.1  but  with  equal  contributions 
during  all  10  years  since  these  are  yearly  costs.  For  example,  the  10-year  present  value 
for  a  $  1  one-time  cost  would  be  calculated  as  follows: 


Present  value  = 


$i 


+ 


$i 


+ 


$i 


(1  +  2.4%)1  (1  +  2.4%)2  (1  +  2.4%)3 


+  ...  + 


$1 


(1  +  2.4%)10 


=  $8.80.  (2.3) 


A  NAVSSES  official  reports  that  6  sites/programs  and  20  users  are  currently  used  to 
input  data  for  HM&E  technical  manuals  into  a  Navy  CSDB,  and  we  assumed  that  an 


2-5 


additional  6  sites/programs  and  20  users  would  be  needed  for  producing  training  courses 
if  the  Bridge  were  implemented  for  the  aggregate  case. 

Some  of  these  additional  licenses  and  fees  might  not  be  additional  costs  to  the  gov¬ 
ernment.  Rather,  they  would  only  be  a  transfer  of  maintenance  funds — from  how  they  are 
already  being  spent  under  current  production  processes  to  how  they  would  be  spent  if  the 
Bridge  were  implemented.  To  the  extent  that  the  added  costs  are  really  a  transfer,  we 
have  overestimated  the  implementation  costs  in  Table  2-3  and  therefore  underestimated 
the  net  present  value  (benefits  less  costs)  of  the  Bridge. 
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3.  Cost  Sayings 


A.  Methodology 

The  cost  savings  for  producing  future  technical  manuals  and  training  courses  using 
the  Bridge  were  calculated  by  estimating  the  savings  for  producing  a  nominal  technical 
manual  and  a  nominal  training  course  and  then  scaling  up  the  results  to  the  production  of 
manuals  and  courses  for  the  aggregate  and  single-system  samples.  Here  are  the  detailed 
steps: 

•  Construct  a  “Current”  flow  chart  of  the  major  processes  now  used  for  producing 
and  delivering  information  from  manufacturers  to  the  Fleet  through  technical 
manuals  and  training  courses  (see  Section  3.B). 

•  Construct  a  “Bridge”  flow  chart  for  the  processes  expected  to  occur  using  the 
Bridge  (see  Section  3.C). 

•  Using  these  flow  charts  as  a  guide,  list  the  detailed  tasks  (see  Appendixes  A  and 
B)  required  to  produce  the  nominal  products:  a  500-page  technical  manual  and  a 
1 -content-hour  training  course.  (The  training  course  is  Interactive  Level  2, 
which  involves  limited  participation  in  that  students  make  simple  responses  to 
instructional  cues.) 

•  Estimate  the  staff  hours  needed  to  perfonn  the  detailed  tasks  for  the  nominal 
products,  currently  and  using  the  Bridge  (see  Appendixes  A  and  B). 

•  Subtract  these  estimates  to  obtain  the  savings  in  staff  hours  to  produce  the 
nominal  products  using  the  Bridge.  Convert  these  savings  to  cost  savings  by 
applying  average  hourly  pay  rates  for  technical  writers  and  course  developers. 
The  savings  are  described  in  Section  3.D.,  along  with  a  description  of  why  they 
occur. 

•  Scale  up  the  cost  savings  for  producing  the  nominal  products  to  estimate  the 
Bridge  savings  for  producing  the  two  samples  described  in  Section  3.E  and 
shown  below.  The  aggregate  sample  consists  of  all  Navy  HM&E  technical 
manuals  and  all  Navy  training  courses  distributed  by  the  NeL  LMS.  The  single¬ 
system  sample  consists  of  the  technical  manuals  and  training  courses  for  the 
AN/AQS-20A  mine-hunting  system  deployed  on  LCS  ships. 
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B.  Current  Processes  Flow  Chart 

Figure  3-1  portrays  the  current  processes  for  producing  technical  manuals  and 
training  courses.  The  processes  cover  the  conversion  of  information  from  standard 
descriptions — normal,  page-turning  documents  written  by  OEM  personnel — to  the  pro¬ 
duction  of  technical  manuals  and  training  courses  and  the  distribution  of  these  materials 
to  the  Fleet.  For  the  production  of  FIM&E  technical  manuals  (left  side),  Navy  personnel 
and  contractors  at  NAVSSES  use  the  OEM  descriptions  to  produce  the  information  in  the 
SGML  markup  language.  (In  some  cases,  the  OEMs  are  tasked  to  provide  the  SGML 
documents.)  NAVSSES  puts  the  documents  into  a  CSDB  repository  and  uses  publication 
tools  to  generate  technical  manuals  in  various  formats  for  distribution  to  the  Fleet.  These 
formats  include  hard-copy  texts,  Interactive  Electronic  Technical  Manuals  (IETMs),  and 
Portable  Document  Format  (PDF)  files.1 


Figure  3-1.  Current  Processes  for  Producing  Technical  Manuals  and  Training  Courses 

The  right  side  of  Figure  3-1  shows  the  current  processes  for  producing  training 
courses.  Contractors  and  government  personnel  who  have  the  skills  described  in 
Table  2-7  use  the  OEM  descriptions,  completed  technical  manuals,  and  the  results  of  job 
task  analyses  to  construct  learning  content.  Authoring  tools  are  used  to  transform  the 


NAVSSES  also  produces  documents  for  Engineering  Operation  Sequencing  Systems  and  Preventive 
Maintenance  Systems,  but  only  the  technical  manuals  are  considered  in  this  study. 
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learning  content  into  instructor-led  and  SCORM-conformant  training  courses.  These  doc¬ 
uments  are  used  with  publication  tools  to  produce  training  course  outputs  such  as  hard¬ 
copy  texts,  digital  versatile  disks  (DVDs),  and  content  for  LMSs. 


C.  Bridge  Flow  Chart 

Figure  3-2  describes  the  major  processes  involved  in  using  the  Bridge.  The  entirely 
new  processes  are  shown  in  blue,  and  the  processes  that  exist  but  are  substantially 
changed  are  shown  in  red.  The  major  differences  from  the  Current  processes  flow  chart 
(see  Figure  3-1)  are  as  follows: 

•  The  S1000D  DM  technical  data  specification  is  used  for  all  text. 

•  All  DMs  are  stored  in  CSDBs. 

-  Course  developers  (1)  use  the  API  to  employ  their  individual  authoring  tools 
to  obtain  technical  DMs  from  the  CSDBs,  (2)  reuse  and  repurpose  the 
information  into  LDMs,  and  (3)  use  the  API  to  store  the  LDMs  back  into  the 
CSDBs.  (“Reuse”  implies  no  change;  “repurpose”  implies  change.) 

•  All  outputs — technical  manuals  and  training  courses — are  produced  from  the 
DMs  and  LDMs. 


•  A  Web  Service  is  used  to  identify  those  DMs  and  LDMs  that  require  review  for 
possible  modification  in  response  to  ECPs. 


New  Production  of  Technical  Manuals 


New  Production  of  Training  Courses 


Standard  Page-Turning  Description 


NAVSSES  (HM&E) 


Legacy  Documents  English,  Book  Form 
Legacy  Documents  SGML,  Book  Form 
New  Documents:  S1000D,  DMs 


English  Book  Form  Documents 
SGML  Book  Form  Documents 
S1000D,  DMs,  LDMs,  PMs 


Technical  Manual  and  Training  Course 
Publication  Tools 


Hard-Copy  Manuals,  IETMS,  and  so  forth 
Hard-Copy  Texts,  LMSs,  and  so  forth 


Figure  3-2.  Processes  for  Producing 
Technical  Manuals  and  Training  Courses  Using  the  Bridge 
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D.  Cost  Savings  for  Producing  the  Nominal  Products 


1.  Calculations 

Table  3-1  shows  the  calculation  of  the  cost  savings  for  the  nominal  products.  The 
number  of  staff  hours  is  discussed  in  the  following  subsection.  The  pay  rate  was  derived 
in  Table  2-4.  The  remainder  of  this  subsection  discusses  the  issue  of  soft  vs.  hard  skill 
training. 


Table  3-1.  Cost  Savings  for  Producing  the 
Nominal  Products  Using  the  Bridge 


500-Page 
Technical  Manual 

1-Content-Hour 
Training  Course 

Number  of  staff  hours 

Current 

9,172 

425 

Bridge 

8,682 

360 

Savings  (hours) 

490  (5.3%) 

65  (15.3%) 

Pay  rate 

Hourly  pay,  fully  burdened 

$65.00 

$65.00 

Total 

$31,850.00 

$4,225.00 

2.  Total  Staff  Hours 

The  Current  and  Bridge  staff  hours  for  the  nominal  technical  manual  are  obtained 
from  the  detailed  38-task  analysis  in  Appendix  A.  The  numbers  for  the  nominal  training 
course  are  obtained  from  the  80-task  analysis  in  Appendix  B.  The  numbers  in  these 
appendixes  were  estimated  by  senior  analysts  who  have  extensive  experience  in  technical 
and  training  analysis. 

The  Bridge  saves  many  more  staff  hours  in  producing  technical  manuals  than  in 
producing  training  courses  (490  vs.  65).  However,  the  percentage  changes  go  in  the 
opposite  direction:  15.3%  saving  for  training  courses  vs.  5.3%  for  technical  manuals." 
The  percentage  differences  are  consistent  with  the  idea  that  the  integration  affects  the 
production  of  training  courses  more  than  that  of  technical  manuals  (see  Figure  3-1  and 
Figure  3-2). 

Listed  below  are  the  principal  reasons  for  the  Bridge  savings  in  producing  the 
nominal  products.  These  reasons  are  also  those  for  the  full  savings  discussed  in  the  next 
section,  which  are  scaled-up  versions  of  the  nominal  savings. 


The  absolute  values  are  not  significant  because  they  simply  reflect  the  different  size  of  the  nominal  man¬ 
ual  and  course  that  the  project  analysts  chose  to  develop. 
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•  Features  of  the  Bridge  that  lead  to  initial  increases  in  staff  hours: 

-  Structuring  the  integration  of  training  and  technical  data 

-  Forming  new  business  rules  to  develop  LDMs 

-  Constructing  a  reuse  strategy 

•  Features  of  the  Bridge  that  lead  to  an  eventual  reduction  in  staff  hours: 

-  Integrated  production,  which  reduces  duplication 

-  Structuring  technical  data  in  DMs  stored  in  centralized  CSDBs,  which 
allows  course  developers  to  avoid  the  time  and  cost  of  collecting  and 
processing  material  from  OEMs  and  other  sources 

-  Organizing  data  by  product-defined  DMs  and  LDMs,  which  makes  it  easier 
to  reuse  technical  and  training  content 

-  Using  the  Web  Service  and  S 1000D,  which  reduces  the  time  needed  to  iden¬ 
tify  the  DMs  and  LDMs  that  require  review  for  modification  in  response  to 
ECPs 

The  5.3%  savings  for  the  technical  manuals  is  consistent  with  recent  NAVSSES 
calculations  based  on  alternative  inputs  to  Table  A-l  in  Appendix  A.  The  NAVSSES  cal¬ 
culations  indicate  that  the  Bridge  would  save  approximately  1.5%  staff  hours  for  a  small 
(10-page  maximum)  technical  manual  for  equipment  updates,  and  percentage  changes 
approximately  three  and  five  times  higher  for  medium  (30-page)  and  full  revision 
(50-page)  updates.  These  numbers  average  5.9%,  using  the  total  number  of  pages  from 
Table  3-2  as  weights:  (1,300  x  1.5%  +  3,900  x  4.5%  +  7,000  x  7.5%)/  (1,300  +  3,900  + 
7,000)  =  5.9%.  The  savings  for  the  larger  manuals  for  new  systems  would  be  even  higher. 
Our  estimate  of  5.3%  savings  thus  understates  the  full  savings. 

E.  Cost  Savings  for  Producing  the  Aggregate  Sample 

1.  Technical  Manuals 

The  cost  savings  for  HM&E  technical  manuals  are  estimated  in  Table  3-3  by  mul¬ 
tiplying  the  savings  per  page  for  the  nominal  500-page  technical  manual  calculated  from 
Table  3-1  by  the  future  yearly  production  of  HM&E  technical  manual  pages  estimated  by 
the  number  of  manual  pages  that  NAVSSES  produced  in  2009  (see  Table  3-2). 

Manuals  are  produced  yearly,  so  the  10-year  present  value  is  calculated  as  in 
Eq.  (2.3).  We  do  not  assume  any  effort  to  systematically  update  all  legacy  manuals.  The 
only  such  manuals  that  are  updated  in  NAVSSES’s  2009  production  are  those  that  are 
required  to  support  current  equipment  upgrades. 
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Table  3-2.  Production  of  Navy  HM&E  Technical  Manuals  in  2009 


Category 

(Nominal 

Number  of  Pages) 

Hours  per 
Technical 
Manual 

Total 

Hourly 

Cost 

Average 
Cost 
(FY10  $) 

Technical 
Manuals 
per  Year 

Total 

Pages 

Updates 

Small  (10) 

23 

$150 

$3,500 

260 

1,300  Note 

Medium  (30) 

67 

$150 

$10,000 

3,900  Note 

Full  Revision  (50) 

167 

$150 

25,000 

140 

7,000 

New 

DDG-1000 

333  (500) 

$150 

$50,000 

67 

33,500 

Total 

- 

- 

- 

- 

45,700 

Note  for  Table  3-2:  The  260  manuals  per  year  were  distributed  evenly  between  small-  and  medium-size 
technical  manuals. 


Table  3-3.  Cost  Savings  From  Producing  HM&E  Technical  Manuals  Using  the  Bridge 


Bridge  savings  in  cost  per  page 

$63.70 

HM&E  technical  manual  pages  produced  per  year 

45,700 

Yearly  cost  savings  (FY10  $) 

$2,911,090 

Occurrences  in  10  years 

10 

10-year  cost  savings,  present  value  (FY10  $) 

$25,610,204 

2.  NeL-Delivered  Training  Courses 

The  cost  savings  for  NeL-delivered  courses  are  calculated  in  Table  3-4  by  mul¬ 
tiplying  the  total  cost  savings  per  content  hour  ($4,225,  see  Table  3-1)  by  the  number  of 
training  content  hours  delivered  by  NeL  in  2009  (3,292,  see  Table  2-6,  with  more  detail 
in  Appendix  E)  and  then  multiplying  the  product  of  this  calculation  by  the  estimated  per¬ 
centage  of  content  hours  involving  hard  skills  (50%).  Hard  skills  are  those  that  are 
needed  to  use  equipment  or  that  otherwise  involve  substantial  technical  content.  They  are 
therefore  the  courses  whose  cost  would  be  directly  reduced  by  integration  of  technical 
and  training  content.  The  training  courses  are  produced  yearly,  so  the  10-year  present 
value  is  calculated  using  Eq.  (2.3),  as  with  the  technical  manuals. 


Table  3-4.  Cost  Savings  From  Producing  NeL  Training  Courses 


Total  cost  savings  per  content  hour  (FY10  $) 

$4,225 

Training  content  hours  delivered  by  NeL  during  2009 

3,292 

Estimated  percentage  of  content  hours  involving  hard  skills 

50% 

One-time  cost  savings  (FY10  $) 

$6,954,350 

Occurrences  in  10  years 

10 

10-year  cost  savings,  present  value  (FY10  $) 

$61,180,632 
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The  estimate  that  50%  of  training  courses  would  involve  hard  skills  was  obtained  by 
inspecting  the  titles  of  NeL  courses  for  mention  of  equipment  or  other  technical  content. 
We  looked  at  the  titles  of  the  450  most  accessed  courses,  a  10%  sample  of  the 
4,504  courses  accessed  at  least  once  during  the  first  5  months  of  FY10.  The  sample  com¬ 
prised  approximately  90%  of  total  accessions.  The  10-year  saving  of  $61.2  million  is  an 
underestimate  since  it  ignores  the  possibility  that  the  Bridge  might  also  achieve  some 
savings  with  soft  skill  courses  by  virtue  of  its  ability  to  organize  data.  We  vary  the  50% 
factor  in  the  sensitivity  analysis  in  Chapter  5. 

F.  Cost  Savings  for  Producing  the  Single-System  Sample 

1.  AN/AQS-20A  Technical  Manuals 

Table  3-5  calculates  the  cost  savings  for  AN/AQS-20A  technical  manuals  by  mul¬ 
tiplying  the  savings  per  page  for  the  nominal  500-page  technical  manual  (see  Table  3-1) 
by  the  number  of  pages  in  the  technical  manuals  of  the  AN/AQS-20A  and  the  MH-60S 
helicopter  (the  detector’s  tow  vehicle).  Recent  Navy  inputs  indicate  that  the  AN/AQS- 
20A  manuals  contain  120  and  210  pages,  respectively.  The  MH-60S  helicopter  manual  is 
reported  to  be  much  larger.  We  will  assume  it  is  500  pages,  for  a  total  of  830  pages  for 
the  three  manuals.  The  10-year  present  value  assumes  the  technical  manual  is  updated 
every  3  years  (produced  in  years  1,  4,  7,  and  10)  to  account  for  upgrades  in  equipment 
and  changes  in  maintenance  procedures.  The  calculation  follows  Eq.  (2.1). 

Table  3-5.  Cost  Savings  From  Producing  AN/AQS-20A  Technical  Manuals  Using  the  Bridge 


Bridge  savings  in  cost  per  page 

$63.70 

Number  of  pages 

830 

Yearly  cost  savings  (FY10  $) 

$52,871 

Occurrences  in  10  years 

4 

10-year  cost  savings,  present  value  (FY10  $) 

$117,645 

2.  AN/AQS-20A  Training  Courses 

Table  3-6  applies  the  costs  savings  per  content  hour  from  Table  3-1  to  the  produc¬ 
tion  of  training  courses  for  the  AN/AQS-20A.  Although  the  2007  Navy  Training  System 
Plan  (NTSP)  stipulates  21  days  (168  hours)  of  schoolhouse  training,  we  assume  that  the 
Navy  would  eventually  transition  to  “blended”  training  of  148  hours  of  school-house 
training  plus  20  hours  for  CBT.  We  assume  that  all  of  the  AN/AQS-20A  courses  train 


3  Table  IV. B. 3  in  the  AN/AQS-20A  Navy  Training  System  Plan  (NTSP)  of  June  2007  lists  two  manuals 
for  the  detector  (A1-H60SA-720-400,  AN/AQS-20A  Organizational  Level  Maintenance  Manual,  and  A1-960QA- 
150-000,  AN/AQS-20A  Intermediate  Level  Maintenance  Manual)  and  one  manual  for  the  helicopter  (Al- 
H60CD-60S-000,  MH-60S  Organizational  Level  Maintenance  Manual ). 
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Table  3-6.  Cost  Savings  From  Producing  AN/AQS-20A  Training  Courses  Using  the  Bridge 


Cost  savings  for  one-content-hour  course 

$4,225 

Planned  course,  C-1 02-01 14  Note 

21  days  of  schoolhouse  training 

Assumed  transition  to  blended  training 

148  hours  schoolhouse;  20  hours  CBT 

Estimated  percentage  of  content  hours  involving  hard  skills 

100% 

One-time  savings  (FY10  $) 

$84,500 

Occurrences  in  10  years 

4 

10-year  cost  savings,  present  value  (FY10  $) 

$188,023 

Note  for  Table  3-6:  See  N85-NTSP-P-30-0305/A,  June  2007,  Navy  Training  System  Plan  for  the  AN/AQS-20A 
Sonar,  Mine  Detecting  Set,  Table  III. A. 2. b. 


hard  skills.  Multiplying  the  20  hours  of  CBT  by  the  savings  per  content  hour  yields  the 
one-time  savings.  The  10-year  present  value  savings  assumes  that  the  training  course 
would  be  updated  every  3  years  (i.e.,  produced  in  years  1,  4,  7  and  10)  to  account  for 
changes  in  technique  and  technology. 
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4.  Contributions  to  Fleet  Readiness 


As  discussed  in  Chapter  1,  the  Bridge  may  improve  Fleet  readiness  and  save  on 
cost.  Technical  manuals  and  training  courses,  unless  they  are  systematically  integrated, 
might  offer  somewhat  different/dated  information,  and  the  delivery  of  training  courses 
might  lag  the  fielding  of  new  systems  and  equipment  upgrades.  These  effects  could 
degrade  the  efforts  of  shipboard  operators  and  maintainers  (and  could  also  add  to  cost  by 
creating  a  need  for  immediate  retraining  of  recently  graduated  trainees). 

The  Navy  has  recently  taken  steps  to  minimize  these  problems  of  disparity  and 
delay  by  adopting  a  policy  of  not  deploying  ships  following  the  installation  of  new  sys¬ 
tems  and  equipment  until  the  updated  support  documents  are  in  place  (see  Appendix  C). 
Implementing  the  Bridge  would  support  this  policy  if  it  shortens  the  time  for  making 
updated  technical  manuals  and  training  courses  available  to  the  Fleet.  We  have  not 
obtained  data  on  the  incidence  of  disparate  and  lagging  infonnation,  but  we  have  carried 
out  a  parametric  calculation  to  detennine  whether  such  delays  (if  they  do  exist)  might 
result  in  a  substantial  loss  of  readiness.  If  they  do,  implementing  the  Bridge  might  offer 
the  Navy  the  substantial  benefit  of  ensuring  against  losses  in  readiness. 

A.  Methodology 

The  main  question  is  how  to  convert  assumed  time  delays  in  delivering  technical 
information  to  the  Fleet  to  losses  in  the  value  of  effectiveness  expressed  in  dollar  terms. 
Say  the  Navy  produces  an  improved  power  supply  for  a  fire-control  system  that  it  fields 
as  an  ECP  equipment  upgrade.  Suppose  that  the  power  supply  has  a  10-year  lifetime  and 
a  unit  cost  of  $10  million  in  amortized  research  and  development  (R&D)  and  procure¬ 
ment.  It  is  ready  for  deployment  in  2010,  and  the  accompanying  technical  information 
(technical  manuals  and  training  courses)  is  also  available  in  2010.  What  is  the  value  of 
the  effectiveness  the  power  supply  would  give  the  Navy?  The  crucial  step  of  the  metho¬ 
dology  is  to  realize  that  the  Navy  would  not  spend  $10  million  on  the  power  supply  unless 
it  thought  that  the  power  supply  was  worth  (at  least)  that  much.  Assuming  that  the 
$10  million  is  spent  this  year  (the  present  value  of  the  investment),  Eq.  (4.1)  regards  the 
$10  million  present  value  as  spending  of  $  1 . 14  million  annually  for  the  power  supply: 

Present  value  -  £i°  $114M  —  $10.00  million.  (4.1) 

^  (1  +  2.4%)n  V  ’ 
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Now,  suppose  technical  manuals  and  training  courses  with  consistent  and  up-to-date 
information  are  not  available  until  2011,  and,  in  accord  with  the  new  policy,  the  ship  sails 
in  2010  but  without  the  new  equipment.  The  present  value  in  2010  is  now  less: 

Present  value  -  YA1  $1,14M  =  $9.79  million.  (4.2) 

The  Navy  therefore  loses  the  effectiveness  the  power  supply  would  have  delivered 
in  2010,  but  now  receives  effectiveness  in  2021.  The  Navy  has  therefore  lost  effective¬ 
ness  that  it  values  at  $0.21  million  or  greater  ($10.00  million  less  $9.79  million).  The  loss 
in  2010  exceeds  the  gain  in  2021  because  the  terms  of  the  sum  are  monotonically 
decreasing.  The  loss  would  be  even  larger  if  we  did  not  discount  but  assumed  the  value  of 
the  power  supply  was  a  constant  $  1  million  per  year.  Thus,  if  integrating  the  production 
of  technical  manuals  and  training  courses  were  to  avoid  the  year  delay  in  delivering  con¬ 
sistent  and  timely  information  to  the  Fleet,  the  Bridge  would  have  achieved  a  10-year 
present-value  benefit  of  $0.21  million. 

B.  The  DDG-1000  Zumwalt  Class  Destroyer 

We  will  now  apply  this  method  to  an  analysis  of  monthly  data  for  the  first  DDG- 
1000  Zumwalt  guided-missile  destroyer.  All  the  components  (electronic,  ordnance,  and 
HM&E)  for  the  first  two  ships  were  funded  at  $2,100  billion  in  FY07  as  listed  in  the 
FY10  President’s  Budget — or  $1,096  billion  per  ship  in  FY10  dollars.  Assuming  the 
components  have  a  10-year  life,  the  Navy  is  paying  approximately  $10.29  million 
monthly  since  the  present  value  of  $10.29  million  monthly  over  120  months  (10  years)  at 
0.20%  monthly  (2.4%  yearly)  yields  the  FY10  present  value  cost  of  $1,096  billion. 

The  120-month  present  value  of  the  value  of  effectiveness  would  be  $1,094  billion 
in  the  case  of  a  1-month  delay  (see  Eq.  (4.3))  and  $1,096  billion  if  the  delay  were  elimi¬ 
nated  (see  Eq.(  4.4)).  Eliminating  the  1 -month  delay  would  therefore  grant  the  Navy  a 
benefit  equal  to  a  $2  million  increase  in  the  value  of  effectiveness: 

Benefit  if  the  logistics  support  is  1  month  late  =  Y\21  $10.29/(1  +  0.2%)n 

=  $1,094  billion.  (4.3) 

Benefit  if  the  logistics  support  is  on  time  =  Yi20  $10.29/(1  +  0.2%)n 

=  $1,096  billion.  (4.4) 

Table  4-1  shows  the  results  for  delays  of  0,  1,  6,  and  12  months.  The  benefit  is 
approximately  $2  million  per  1 -month  delay  of  technical  infonnation  for  all  components 
per  ship.  Ensuring  against  lags  might  therefore  have  significant  value  for  readiness  and 
suggests  that  an  empirical  analysis  should  be  conducted  using  actual  Fleet  data. 
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Table  4-1.  Bridge  Reductions  in  Delays  of  Delivering  Technical  Information  to  the  Fleet 

Delay  in  Availability  of  New  Systems  or  Equipment 
1  Month  6  Months 

$1,094  billion  $1 .083  billion 
$2  million  $13  million 


12  Months 

$1,070  billion 
$26  million 


Month 


$1 .096  billion 


$0 


5.  Summary 


This  chapter  summarizes  the  analysis:  the  Base  Case,  a  sensitivity  analysis  of  the 
major  uncertainties,  future  work,  and  overall  implications. 

A.  Quantitative  Results  Base  Case 

Table  5-1  summarizes  the  quantitative  results  of  the  analysis.  The  analysis  of  both 
samples  support  the  attractiveness  of  the  Bridge. 


Table  5-1.  Summary  of  Cost-Benefit  Analysis 


10-Year  Present  Value 
(FY10  $) 

Technical 

Manuals 

Training 

Courses 

Total 

Aggregate  Sample  (the  OSD  Perspective) 

Cost 

First  year  investment  (see  note  1) 

$1.8  million 

Second  year  investment 

$1 .8  million 

Implementation  cost 

$2.9  million 

$4.0  million 

$6.9  million 

Total  cost  second  year 

$8.7  million 

Benefits 

$25.6  million 

$61 .2  million 

$86.8  million 

Net  present  value 

$78.1  million 

Parametric  analysis  of  readiness 

Note  2 

Single-System  Sample  (the  Program  Office  Perspective) 

Benefits 

$117,600 

$188,000 

$305,700 

Note  1:  These  are  sunk  costs  and  therefore  are  not  included  in  total  (future)  cost. 

Note  2:  Loss  of  $2  million  worth  of  effectiveness  per  1 -month  delay  of  technical  information  for  all  compo¬ 
nents  per  ship. 


For  the  aggregate  analysis  (the  OSD  perspective),  the  $86.8  million  present  value  of 
the  benefits  of  the  Bridge  more  than  covers  the  $8.7  million  present  value  of  the  cost  for  a 
sizable  portion  of  the  Navy’s  technical  manuals  and  training  courses.  These  savings  are 
approximately  9.9%  of  the  estimated  current  10-year  present  value  of  producing  these 
manuals  and  courses,  using  the  methodology  employed  in  this  study.  The  net  present 
value  (benefits  less  costs)  of  $78.1  million  would  be  even  higher  if  the  Bridge  were  used 
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The  benefits  of  the  AN/AQS-20A  analysis  (the  Program  Office  perspective)  are 
almost  $306,000,  or  a  savings  of  approximately  8.9%  of  the  current  10-year  present 
value.  As  we  said  in  Chapter  1  Section  E.2,  we  are  ignoring  the  costs  for  this  case  since  it 
is  not  reasonable  to  compare  the  full  investment  and  implementation  costs  of  the  Bridge 
with  the  benefits  of  implementing  it  for  just  a  single  system.  The  benefits  could,  of 
course,  be  compared  with  only  the  implementation  costs,  which  are  likely  to  be  a  small 
fraction  of  those  for  the  much  larger  aggregate  sample.  The  workloads  for  the  AN/AQS- 
20A  sample  (number  of  pages  of  technical  manuals  and  number  of  content  hours  of 
training  courses)  are  only  approximately  1%  of  those  for  the  aggregate  case.  Therefore,  if 
the  implementation  costs  scale  as  the  workload  scales,  they  would  be  less  than  $70,000 
(1%  of  $6.9  million) — much  less  than  the  benefit  of  $306,000. 

B.  Sensitivity  Analysis  of  the  HM&E/NETC  Case 

As  noted  earlier,  the  analysis  in  this  study  depends  on  the  values  of  several  uncer¬ 
tain  inputs.  This  section  discusses  the  effect  of  these  inputs  on  the  base  case  results  of  the 
aggregate  case  shown  in  Table  5-1.  The  results  of  this  analysis  are  shown  in  Table  5-2. 
The  first  two  cases  yield  higher  net  benefit  than  the  Base  Case,  and  the  three  cases  fol¬ 
lowing  the  Base  Case  yield  lower  cost.  This  table  also  shows  the  net  benefits  from  taking 
the  variations  in  combination. 1 

1.  50%  Higher  Pay  Rate 

The  $65  fully  burdened  hourly  pay  rate  used  in  Table  2-4  is  close  to  the  rates 
actually  observed  for  recent  technical  manuals  and  training  courses.  A  higher  rate  might 
be  needed  to  attract  people  trained  in  using  the  Bridge.  We  tried  using  $97.50,  which  is 
50%  higher.  (This  rate  is  approximately  equal  to  the  burdened  rate  for  a  GS-12,  which 
seems  more  than  sufficient.)  This  new  rate  increases  the  implementation  costs  because  of 
higher  training  costs,  but  the  benefits  in  lowering  the  staff  hours  to  produce  future 
manuals  and  courses  rise  much  more,  leading  to  a  much  higher  net  benefit:  $119.6  mil¬ 
lion  compared  to  $78.1  million,  a  rise  of  53%. 


Another  uncertainty  is  the  size  of  the  aggregate  samples.  Although  we  lack  quantitative  analysis,  its 
effect  on  net  benefit  is  clearly  large  and  positive.  The  Navy  produces  technical  manuals  for  electronics 
and  ordnance  equipment — not  just  those  for  HM&E — and  training  courses  delivered  by  LMSs  and  those 
delivered  by  NeL.  In  addition,  the  Navy  would  not  be  the  only  Service  to  realize  the  benefits  of  using  the 
Bridge.  The  methodology  of  the  aggregate  analysis  is  quite  general  and  would  also  lead  to  net  benefits 
for  the  other  Services. 
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Table  5-2.  Sensitivity  Analysis 


10-Year  Present  Value  (FY10  $) 

Costs 

Benefits 

Change  From 

Base  Case 

Investment 

Implementation 

Total 

Technical  Manuals 

Training  Courses 

Total 

Net  Benefit 

1 .  50%  higher  pay  rate 

$1.8  M 

$8.8  M 

$10.6  M 

$38.4  M 

$91.8  M 

$130.2  M 

$119.6  M 

2.  50%  more  hard  skills 

$1.8  M 

$6.9  M 

$8.7  M 

$25.6  M 

$91.8  M 

$117.4  M 

$108.7  M 

Base  Case 

$1.8  M 

$6.9  M 

$8.7  M 

$25.6  M 

$61.2  M 

$86.8  M 

$78.1  M 

3.  100%  higher  invest¬ 
ment  cost 

$3.6  M 

$6.9  M 

$10.5  M 

$25.6  M 

$61.2  M 

$86.8  M 

$76.3  M 

4.  200%  higher  imple¬ 
mentation  cost 

$1.8  M 

$20.6  M 

$22.4 

$25.6  M 

$61.2.  M 

$86.8  M 

$64.4  M 

5.  50%  lower  training 
course  savings 

$1.8  M 

$6.9  M 

$8.7  M 

$25.6  M 

$30.6  M 

$56.2  M 

$47.5  M 

6.  Combination  of  3-5 

$3.6  M 

$20.7  M 

$24.2  M 

$25.6  M 

$30.6  M 

$56.2  M 

$32.0  M 

7.  Combination  of  1-5 

$3.6  M 

$26.4  M 

$30.0  M 

$38.4  M 

$68.8  M 

$107.2  M 

$77.3  M 

8.  Combination  of  1  and  2 

$1.8  M 

$8.8  M 

$10.6  M 

$38.4  M 

$137.7  M 

$176.1  M 

$165.5  M 

2.  50%  More  Hard-Skill  Courses 

Table  3-4  assumed  that  50%  of  NeL  training  courses  involved  substantial  technical 
data  and  would  therefore  show  direct  cost  savings  from  integrating  technical  manuals  and 
training  courses.  This  percentage  was  obtained  by  counting  the  number  of  titles  of  NeL- 
delivered  courses  that  mentioned  equipment  or  other  technical  data.  We  looked  at  10%  of 
the  titles —  a  10%  sample  that  comprises  90%  of  total  accessions.  Applying  the  50%  to 
the  entire  cost  savings,  however,  produces  an  underestimate  of  the  cost  savings  because  it 
does  not  consider  the  fact  that  courses  used  to  train  hard  skills  are  usually  longer  and 
more  likely  to  receive  upgrades  and  would  thus  benefit  more  from  integration.  Moreover, 
it  ignores  the  benefits  for  producing  soft-skill  courses  (e.g.,  leadership)  because  of  the 
Bridge’s  greater  organization  of  information  and  the  savings  of  using  the  Web  Service  for 
making  upgrades.  The  sensitivity  analysis  estimates  these  effects  by  assuming  that  75% 
of  courses  train  hard  skills.  The  net  benefit  would  rise  from  $78.1  million  to  $108.7  mil¬ 
lion,  a  rise  of  39%. 

3.  100%  Higher  Investment  Cost 

Investment  cost  would  rise  if  it  takes  longer  than  the  second  year  of  the  RTOC 
project  to  develop  the  Bridge  software  to  the  point  that  it  is  ready  for  implementation  by 
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the  Navy.  We  assumed  that  a  third  year  would  be  required,  resulting  in  a  50%  increase  in 
total  cost  (from  2  to  3  years).  However,  this  additional  time  would  cause  a  100%  increase 
in  investment  cost,  since  the  first  year  is  sunk  cost  and  not  considered  in  the  calculations. 
The  investment  cost  is  much  smaller  than  the  benefits,  so  net  benefit  would  fall  by  only 
2%,  or  less  than  $2  million. 

4.  200%  Higher  Implementation  Cost 

Implementing  the  Bridge  would  require  a  cultural  shift  in  the  technical  manual  and 
training  course  workforce,  which  could  increase  all  three  components  of  implementation 
cost:  trainee  personnel  cost  (increased  training  time),  trainee  CBT  course  (more  extensive 
learning  content),  and  site/program  licenses  and  user  fees  (more  programmers  being 
involved).  We  have  assumed  a  200%  increase,  or  tripling  of  implementation  cost.  Net 
benefit  would  fall  from  $78. 1  million  to  $64.4  million,  an  18%  decrease. 

5.  50%  Lower  Training  Course  Savings 

The  analysis  of  Bridge  savings  detailed  in  Appendixes  A  and  B  and  summarized  in 
Table  3-1  found  a  5.3%  reduction  in  staff  hours  and  cost  for  producing  technical  manuals 
and  a  much  larger  15.3%  reduction  for  training  courses.  The  larger  reduction  for  training 
courses  is  understandable,  given  that  integration  will  permit  course  developers  to  use  the 
API  to  access  technical  data  stored  in  DMs  in  CSDBs.  However,  if  these  savings  proved 
smaller  by  half,  net  benefit  would  fall  from  $78.1  million  to  $47.5  million,  or  a  39% 
decrease. 

6.  Combined  Effects 

The  last  three  rows  of  Table  5-2  show  that  combining  the  variations  leads  to  a  range 
of  net  benefit  varying  from  $32.0  million  (a  59%  decrease  relative  to  the  Base  Case)  to 
$165.5  million  (a  1 12%  increase  relative  to  the  Base  Case): 

•  $32.0  million  if  the  three  negative  inputs  (items  3-5)  and  none  of  the  positive 
inputs  were  to  occur  simultaneously 

•  $77.3  million  if  all  five  inputs  occurred  simultaneously  (coincidentally  close  to 
the  $78. 1  million  for  the  Base  Case) 

•  $165.5  million  if  only  the  two  positive  inputs  (items  1  and  2)  occurred 

C.  Future  Work 

This  study  develops  a  methodology  for  analyzing  the  effects  of  the  Bridge,  and  it 
provides  initial  quantitative  results  that  are  planned  to  be  refined  in  the  second  year  of  the 
project  by  these  steps: 
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•  Validating  the  analysis  of  cost  savings  by  more  detailed  analysis  and  by 
obtaining  inputs  from  other  professionals  in  the  technical  infonnation 
community 

•  Discriminating  between  the  cost  savings  for  new  systems  and  equipment 
upgrades  via  ECPs 

•  Obtaining  data  on  the  current  delays  of  providing  timely  technical  infonnation 
to  the  Navy  to  estimate  the  readiness  benefit  of  the  Bridge 

D.  Overall  Summary  and  Implications 

Our  analysis  suggests  that  the  Navy  could  realize  substantial  savings  by  integrating, 
or  bridging,  the  future  production  of  technical  manuals  and  training  courses.  We  found 
that  10-year  savings  (benefits  less  costs)  ranged  from  $32.0  million  to  $165.5  million  for 
integrating  the  yearly  production  of  all  HM&E  technical  manuals  and  CBT  courses 
delivered  by  NeL. 

The  single-system  analysis  found  that  applying  the  Bridge  to  the  AN/AQS-20A 
mine-hunting  sonar  for  the  LCS  would  produce  10-year  benefits  of  $307,700.  These 
would  be  net  savings  if  the  full  investment  and  implementation  costs  were  already  paid. 
If  not,  it  would  take  5.8  years  to  cover  the  $1.8  million  cost  of  investment,  plus  some 
additional  years  to  cover  the  small  but  uncalculated  implementation  costs  for  the 
AN/AQS-20A  alone. 

The  integration  would  be  achieved  through  new  processes  and  software  developed 
by  the  Bridge  project:  adopting  a  common  data  format  (S1000D  DMs)  and  repository 
(CSDB),  using  the  new  API  to  read  the  data,  and  the  new  Web  Service  to  identify  the 
DMs  that  would  require  modification  by  ECPs. 

The  resulting  strategic  collaboration  between  the  technical  and  training  communities 
could  also  produce  other  efficiencies,  such  as  improved  consistency  and  timeliness  of  the 
manuals  and  courses. 
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Appendix  A 

Detailed  Staff-Hour  Estimates  for  Producing 
a  Nominal  500-Page  Technical  Manual 
Under  Current  and  Bridge  Processes 


Table  A-l  compares  the  staff-hours  under  current  and  Bridge  processes  for  producing  the  nominal 
500-page  technical  manual.  The  hours  are  performed  by  contractors  (Cont.)  and  government  (Govt.) 
personnel.  The  definitions  for  the  acronyms  used  in  the  table  are  in  the  list  of  abbreviations  at  the  end  of 
this  report. 


Table  A-1.  Technical  Manual  Staff-Hours:  Current  and  Bridge 


Current 

Bridge 

Task 

Cont. 

Govt. 

Total 

Cont. 

Govt. 

Total 

Changes 

Notes 

Definition 

Obtain  LSAR  data  (if  available) 

240 

80 

320 

240 

80 

320 

Obtain  OEM  content 

240 

40 

280 

240 

40 

280 

Obtain  engineering  drawings 

320 

80 

400 

320 

80 

400 

1 

Collect  provisioning  data 

320 

80 

400 

320 

80 

400 

Obtain  preventive  maintenance  data 

320 

40 

360 

320 

40 

360 

Identify  special  support  and  test  equipment 
requirements 

160 

40 

200 

160 

40 

200 

Identify  source  data  formats 

40 

8 

48 

40 

8 

48 

2 

Specify  deliverable  formats  (IETM/ 
hardcopy) 

40 

8 

48 

40 

8 

48 

3 

Construct  the  functionality  matrix 

120 

20 

140 

120 

20 

140 

Register  with  TDMIS 

8 

8 

8 

8 

4 

Identify  any  conversion  issues 

80 

80 

80 

80 

Conduct  a  guidance  and  planning 
conference 

160 

120 

280 

160 

120 

280 

Resolve  any  NIAPS/TDKM/N  MCI/AT  IS 
issues 

80 

8 

88 

80 

8 

88 

Development 

Develop  and  update  a  book  plan 

0 

0 

Commence  content  development 

0 

0 

Business  rules 

480 

120 

600 

520 

140 

660 

-60 

1 

Reusable  objects/strategy 

160 

0 

160 

0 

160 

2 
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Table  A-1.  Technical  Manual  Staff-Hours:  Current  and  Bridge  (Continued) 


Current 

Bridge 

Task 

Cont. 

Govt. 

Total 

Cont. 

Govt. 

Total 

Changes 

Notes 

Development  (Continued) 

Search  ADL-R 

0 

0 

0 

1 

1 

-1 

Develop  a  DMC/SNS  strategy 

200 

40 

240 

200 

40 

240 

Develop  DMRL  (Training  DMRL) 

160 

40 

200 

160 

40 

200 

Coordinate  with  training  activities 

10 

16 

56 

8 

4 

12 

44 

3 

Produce  a  review  draft  copy 

1,280 

1,280 

1,280 

1,280 

Produce  a  preliminary  draft  copy 

960 

960 

960 

960 

Produce  final  copy 

960 

960 

960 

960 

Develop  PMs 

80 

16 

98 

80 

16 

96 

Perform  three  QA  reviews 

0 

0 

Conduct 

440 

360 

800 

440 

360 

800 

Validate 

240 

80 

320 

240 

80 

320 

Verify 

160 

200 

360 

160 

200 

360 

Delivery 

Produce  CDs  and  hardcopy  manuals 

Coordinate  delivery  with  SNIP  production 
modules 

Life-Cycle  Maintenance 

Modify  technical  information  in  response  to 
ECPs 

0 

0 

Technical  data  DMs 

80 

80 

1 

1 

79 

4 

Training  DMs 

40 

40 

1 

1 

39 

4 

PMS  DMs 

40 

40 

1 

1 

39 

4 

Parts  DMs 

80 

80 

1 

1 

79 

4 

Testing  DMs 

40 

40 

1 

1 

39 

4 

Update  links  with  training  products 

80 

80 

8 

8 

72 

4,  5 

Total 

7,768 

1,404 

9,172 

7,270 

1,412 

8,682 

490 

Note  1  for  Table  A-1:  Additional  Business  Rules  are  needed  to  develop  Learning  Data  Modules  (LDMs)  and  to  structure  the 
interaction  of  training  content  with  technical  data  content. 


Note  2  for  Table  A-1:  Although  more  time  will  be  needed  initially  to  document  a  reusable  objects  strategy,  the  use  of  Data 
Modules  (DMs)  will  eventually  save  time  by  making  the  reuse  of  technical  and  training  content  more  convenient. 

Note  3  for  Table  A-1:  Once  the  processes  to  coordinate  the  production  of  technical  data  and  training  content  are  initially 
established,  these  processes  should  lead  to  future  cost  savings. 

Note  4  for  Table  A-1 :  Engineers  should  no  longer  have  to  spend  large  amounts  of  time  discovering  which  DMs  and  LDMs  will 
require  review  for  possible  modification  in  response  to  Engineering  Change  Proposals  (ECPs). 

Note  5  for  Table  A-1:  The  granularity  of  data  in  DMs  should  lead  to  faster  updating  (i.e. ,  less  time  to  make  the  modifications). 
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Appendix  B 

Detailed  Staff-Hour  Estimates  for 
Producing  a  Nominal  1-Content-Hour  Training  Course 
Under  Current  and  Bridge  Processes 


Table  B-l  shows  detailed  staff-hour  estimates  for  producing  the  nominal  one-content-hour  training 
course  under  current  processes,  and  Table  B-2  shows  the  estimates  using  the  Bridge.  Table  B-3  high¬ 
lights  the  changes.  The  definitions  for  the  acronyms  used  in  the  tables  are  in  the  list  of  abbreviations  at 
the  end  of  this  report. 


Table  B-1.  Training  Course  Staff-Hours:  Current  Processes 


Government 

Government  or  Contractor 

Task 

Training  Director 

Rate  Lead  (SME) 

Learning  Standards  Officer 

Content  Sponsor 

Course  Supervisor 

Contracting  Officer 

Instructional  Systems  Designer 

Graphics 

Animation 

Programmer 

Editor 

Lead 

Total 

Identify  training  requirements 

0 

Identify  training  product  and  training  environment;  review 
OEM  descriptions  of  systems  and  equipment 

2 

2 

Specify  training  content  requirements  (through  job  duty 
task  analysis  and  other  resources) 

1 

1 

Initiate  Navy  Enlisted  classification  request  (if  required) 

1 

1 

Designate  sites  where  training  will  occur 

1 

1 

Designate  course  curriculum  model  manager  (if  required) 

1 

1 

Request  CIN/CDP 

1 

1 

Construct  learning  objective  statements 

1 

1 

2 

Sequence  learning  objective  statements 

1 

1 

2 

Determine  assessment  strategy 

1 

1 

2 

Determine  training  delivery  method 

1 

1 

2 
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Table  B-1.  Training  Course  Staff-Hours:  Current  Processes  (Continued) 


Government 

Government  or  Contractor 

Task 

Training  Director 

Rate  Lead  (SME) 

Learning  Standards  Officer 

Content  Sponsor 

Course  Supervisor 

Contracting  Officer 

Instructional  Systems  Designer 

Graphics 

Animation 

Programmer 

Editor 

Lead 

Total 

Identify  training  resource  requirements 

1 

1 

Identify  instructor  requirements 

0 

Identify  training  device  requirements 

0 

Identify  classroom  and  lab  requirements 

0 

Access  the  CSDB  via  the  API  to  search  for  reusable  DMs 

0 

Search  existing  material 

4 

4 

Determine  level  of  reuse  between  technical  data  and 
training  data  modules 

0 

Create  course  outline  of  instruction 

1 

1 

2 

Develop  an  initial  list  of  required  LDMs 

0 

Determine  evaluation  plan  (formative  and  summative 
evaluations) 

1 

1 

2 

Develop  training  project  plan 

1 

1 

2 

Assemble  resources 

0 

Prepare  acquisition/production  package  (if  contracting) 

1 

1 

Specify  deliverables  (LDMs,  SCORM  content  package 
module,  training  products) 

0 

Select  the  team  (SMEs  and  so  forth) 

1 

1 

1 

3 

Kick  off  the  project 

1 

1 

1 

1 

1 

1 

1 

2 

9 

Hold  technical  data/training  planning  conference 

0 

Hold  learning  objective  statement  conference 

1 

1 

1 

1 

4 

Announce  course  delivery  dates  (content  announcement 
form) 

0 

Create  instructional  media  design  package 

4 

4 

Coordinate,  establish,  and  agree  on  standard  numbering 
system  for  project 

0 

Review  the  instructional  media  design  package  and 
agree  on  business  rules 

1 

1 

Create  test  package 

1 

1 

2 

4 
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Table  B-1.  Training  Course  Staff-Hours:  Current  Processes  (Continued) 


Government 

Government  or  Contractor 

Task 

Training  Director 

Rate  Lead  (SME) 

Learning  Standards  Officer 

Content  Sponsor 

Course  Supervisor 

Contracting  Officer 

Instructional  Systems  Designer 

Graphics 

Animation 

Programmer 

Editor 

Lead 

Total 

Create  content 

0 

Prepare  content  prototype 

1 

2 

1 

4 

Develop  and  obtain  approval  of  DM  requirements  list  for 
LDMs  and  develop  beta  training  products  in  runtime 
environment 

0 

Get  DMs  from  the  CSDB  via  the  API 

0 

Design  interface  and  controls 

1 

1 

1 

3 

Design  for  accessibility 

2 

2 

Comply/conform  to  SCORM 

2 

2 

Review  ILE  content  prototype 

1 

1 

2 

Construct  assessments  (i.e.,  test  bank) 

1 

1 

1 

3 

Load  LDMs  into  CSDB 

0 

Develop  test  design  (i.e.,  length,  question  type,  location, 
and  so  forth) 

1 

1 

2 

4 

Develop  storyboards,  trainees  guides  and  lesson  plans 

0 

Storyboards 

29 

39 

16 

84 

Trainee  guides 

5 

1 

10 

16 

Lesson  plans,  instructor  guides  and  exercise 
controller  guides 

5 

1 

10 

16 

Portion-mark  content 

5 

20 

1 

26 

Review  storyboards,  trainees  guides,  and  lesson  plans 

2 

1 

3 

Construct  assets  (i.e.,  media  and  so  forth) 

86 

11 

8 

7 

112 

Construct  enabling  learning  objects 

1 

1 

2 

Construct  terminal  learning  objects 

2 

1 

3 

Develop  work  flow;  review  and  approve  LDMs  and 
training  products 

0 

Translate  LDMs  into  training  products  for  display  in  Web- 
based  browser 

0 

Check  LDMs  in  CSDB 

0 
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Table  B-1.  Training  Course  Staff-Hours:  Current  Processes  (Continued) 


Government 

Government  or  Contractor 

Task 

Training  Director 

Rate  Lead  (SME) 

Learning  Standards  Officer 

Content  Sponsor 

Course  Supervisor 

Contracting  Officer 

Instructional  Systems  Designer 

Graphics 

Animation 

Programmer 

Editor 

Lead 

Total 

Review  and  accept  learning  objects 

2 

2 

4 

Publish  content  in  LMS 

1 

10 

11 

Publish  content  in  LMS  using  API 

0 

Conduct  government  acceptance  testing 

1 

1 

2 

4 

Submit  content  submission  form  on  content  forecasting 
system 

1 

1 

2 

Deliver  course 

0 

Execute  pre-pilot  checklist 

1 

1 

2 

Perform  pre-pilot  conference 

1 

1 

2 

Set  up  classroom 

1 

1 

Deliver  course  documents 

1 

1 

Conduct  Train-the-Trainer 

1 

1 

Identify  pilot  participants 

1 

1 

Perform  pilot 

2 

2 

Execute  post  pilot  checklist 

1 

1 

Finalize  training  course  control  document 

1 

1 

Perform  content  corrections 

1 

1 

1 

1 

1 

1 

1 

1 

8 

Perform  post  pilot  conference 

1 

1 

Recommend  promulgation 

1 

1 

2 

Promulgate 

1 

1 

Manage  content 

Review  ILE  content 

1 

1 

1 

3 

Register  SCORM-conformant  courseware  with  ADL-R 

1 

1 

Modify  technical  information  in  response  to  ECPs 

40 

40 

Total 

2 

90 

29 

4 

8 

1 

43 

168 

14 

31 

29 

6 

426 
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Table  B-2.  Training  Course  Staff-Hours:  Bridge  Processes 


Government 

Government  or  Contractor 

Task 

Training  Director 

Rate  Lead  (SME) 

Learning  Standards  Officer 

Content  Sponsor 

Course  Supervisor 

Contracting  Officer 

Instructional  Systems  Designer 

Graphics 

Animation 

Programmer 

Editor 

Lead 

Total 

Identify  training  requirements 

0 

Identify  training  product  and  training  environment;  review 
OEM  descriptions  of  systems  and  equipment 

1 

1 

Specify  training  content  requirements  (through  job  duty 
task  analysis  and  other  resources) 

1 

1 

Initiate  Navy  Enlisted  Classification  request  (if  required) 

1 

1 

Designate  sites  where  training  will  occur 

1 

1 

Designate  course  curriculum  model  manager  (if  required) 

1 

1 

Request  CIN/CDP 

1 

1 

Construct  learning  objective  statements 

1 

1 

2 

Sequence  learning  objective  statements 

1 

1 

2 

Determine  assessment  strategy 

1 

1 

2 

Determine  training  delivery  method 

1 

1 

2 

Identify  training  resource  requirements 

1 

1 

Identify  instructor  requirements 

0 

Identify  training  device  requirements 

0 

Identify  classroom  and  lab  requirements 

0 

Access  the  CSDB  via  the  API  to  search  for  reusable  DMs 

1 

1 

Search  existing  material 

0 

0 

Determine  level  of  reuse  between  technical  data  and 
training  data  modules 

1 

1 

Create  course  outline  of  instruction 

0 

Develop  an  initial  list  of  required  LDMs 

1 

1 

2 

Determine  evaluation  plan  (formative  and  summative 
evaluations) 

1 

1 

2 

Develop  training  project  plan 

1 

1 

2 
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Table  B-2.  Training  Course  Staff-Hours:  Bridge  Processes  (Continued) 


Government 

Government  or  Contractor 

Task 

Training  Director 

Rate  Lead  (SME) 

Learning  Standards  Officer 

Content  Sponsor 

Course  Supervisor 

Contracting  Officer 

Instructional  Systems  Designer 

Graphics 

Animation 

Programmer 

Editor 

Lead 

Total 

Assemble  resources 

0 

Prepare  acquisition/production  package  (if  contracting) 

0 

Specify  deliverables  (LDMs,  SCORM  content  package 
module,  training  products) 

1 

1 

Select  the  team  (SMEs  and  so  forth) 

1 

1 

1 

3 

Kick  off  the  project 

0 

Hold  technical  data/training  planning  conference 

1 

1 

1 

1 

1 

1 

1 

2 

9 

Hold  learning  objective  statement  conference 

1 

1 

1 

3 

Announce  course  delivery  dates  (Content  Announcement 
Form) 

1 

1 

Create  instructional  media  design  package 

0 

Coordinate,  establish,  and  agree  on  standard  numbering 
system  for  project 

4 

4 

Review  the  instructional  media  design  package  and 
agree  on  business  rules 

1 

1 

Create  test  package 

1 

1 

2 

4 

Create  content 

0 

Prepare  content  prototype 

0 

Develop  and  obtain  approval  of  DM  requirements  list  for 
LDMs  and  develop  beta  training  products  in  runtime 
environment 

1 

1 

1 

3 

Get  DMs  from  the  CSDB  via  the  API 

1 

1 

Design  interface  and  controls 

1 

1 

1 

3 

Design  for  accessibility 

2 

2 

Comply/conform  to  SCORM 

0 

0 

Review  ILE  content  prototype 

1 

1 

2 

Construct  assessments  (i.e.,  test  bank) 

0 

Load  LDMs  into  CSDB 

1 

1 

1 

3 
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Table  B-2.  Training  Course  Staff-Hours:  Bridge  Processes  (Continued) 


Government 

Government  or  Contractor 

Task 

Training  Director 

Rate  Lead  (SME) 

Learning  Standards  Officer 

Content  Sponsor 

Course  Supervisor 

Contracting  Officer 

Instructional  Systems  Designer 

Graphics 

Animation 

Programmer 

Editor 

Lead 

Total 

Develop  test  design  (i.e. ,  length,  question  type,  location, 
and  so  forth) 

1 

1 

2 

4 

Develop  storyboards,  trainees  guides  and  lesson  plans 

0 

Storyboards 

19 

39 

10 

68 

Trainee  guides 

5 

1 

10 

16 

Lesson  plans,  instructor  guides,  and  exercise 
controller  guides 

5 

1 

10 

16 

Portion-mark  content 

5 

20 

1 

26 

Review  storyboards,  trainees  guides  and  lesson  plans 

2 

1 

3 

Construct  assets  (i.e.,  media  and  so  forth) 

86 

11 

8 

7 

112 

Construct  enabling  learning  objects 

1 

1 

2 

Construct  terminal  learning  objects 

1 

1 

1 

Develop  work  flow;  review  and  approve  LDMs  and 
training  products 

1 

1 

2 

4 

Translate  LDMs  into  training  products  for  display  in  Web- 
based  browser 

1 

3 

4 

Check  LDMs  in  CSDB 

1 

1 

Review  and  accept  learning  objects 

1 

1 

2 

Publish  content  in  LMS 

0 

Publish  content  in  LMS  using  API 

1 

1 

2 

Conduct  government  acceptance  testing 

1 

1 

1 

1 

Submit  content  submission  form  on  content  forecasting 
system 

1 

1 

2 

Deliver  course 

0 

Execute  pre-pilot  checklist 

1 

1 

2 

Perform  pre-pilot  conference 

1 

1 

2 

Set  up  classroom 

1 

1 
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Table  B-2.  Training  Course  Staff-Hours:  Bridge  Processes  (Continued) 


Government 

Government  or  Contractor 

Task 

Training  Director 

Rate  Lead  (SME) 

Learning  Standards  Officer 

Content  Sponsor 

Course  Supervisor 

Contracting  Officer 

Instructional  Systems  Designer 

Graphics 

Animation 

Programmer 

Editor 

Lead 

Total 

Deliver  course  documents 

1 

1 

Conduct  Train-the-Trainer 

1 

1 

Identify  pilot  participants 

1 

1 

Perform  pilot 

2 

2 

Execute  post  pilot  checklist 

1 

1 

Finalize  training  course  control  document 

1 

1 

Perform  content  corrections 

1 

1 

1 

1 

1 

1 

1 

1 

8 

Perform  post  pilot  conference 

1 

1 

Recommend  promulgation 

1 

1 

2 

Promulgate 

1 

1 

Manage  content 

0 

Review  ILE  content 

1 

1 

1 

3 

Register  SCORM-conformant  courseware  with  ADL-R 

1 

1 

Modify  technical  information  in  response  to  ECPs 

1 

1 

Total 

2 

48 

29 

5 

8 

1 

33 

168 

14 

24 

23 

5 

360 
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Table  B-3.  Changes  in  Staff-Hours  To  Produce  Training  Courses  Using  the  Bridge 


Task 

Current 

Bridge 

Change 

Notes 

Identify  training  requirements 

0 

0 

Identify  training  product  and  training  environment;  review 
OEM  descriptions  of  systems  and  equipment 

2 

1 

1 

1 

Specify  training  content  requirements  (through  job  duty 
task  analysis  and  other  resources) 

1 

1 

Initiate  Navy  Enlisted  Classification  request  (if  required) 

1 

1 

Designate  sites  where  training  will  occur 

1 

1 

Designate  course  curriculum  model  manager  (if  required) 

1 

1 

Request  CIN/CDP 

1 

1 

Construct  learning  objective  statements 

2 

1 

1 

2 

Sequence  learning  objective  statements 

2 

2 

Determine  assessment  strategy 

2 

2 

Determine  training  delivery  method 

2 

2 

Identify  training  resource  requirements 

1 

1 

Identify  instructor  requirements 

0 

0 

Identify  training  device  requirements 

0 

0 

Identify  classroom  and  lab  requirements 

0 

0 

Access  the  CSDB  via  the  API  to  search  for  reusable  DMs 

0 

1 

Search  existing  material 

4 

0 

4 

3 

Determine  level  of  reuse  between  technical  data  and 
training  data  modules 

0 

1 

-1 

1 

Create  course  outline  of  instruction 

2 

0 

2 

Develop  initial  list  of  required  LDMs 

0 

2 

-2 

Determine  evaluation  plan  (formative  and  summative 
evaluations) 

2 

2 

Develop  training  project  plan 

2 

2 

Assemble  resources 

0 

0 

Prepare  acquisition/production  package  (if  contracting) 

1 

0 

1 

Specify  deliverables  (LDMs,  SCORM  content  package 
module,  training  products) 

0 

1 

-1 

Select  the  team  (SMEs  and  so  forth) 

3 

3 

Kick  off  the  project 

9 

0 

9 

Hold  technical  data/training  planning  conference 

0 

9 

-9 

Hold  learning  objective  statement  conference 

4 

3 

1 

Announce  course  delivery  dates  (Content  Announcement 
Form) 

1 

1 

Create  instructional  media  design  package 

4 

0 

4 
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Table  B-3.  Changes  in  Staff-Hours  To  Produce  Training  Courses  Using  the  Bridge  (Continued) 


Task 

Current 

Bridge 

Change 

Notes 

Coordinate,  establish,  and  agree  on  standard  numbering 
system  for  project 

0 

4 

-4 

Review  the  instructional  media  design  package  and 
agree  on  business  rules 

2 

2 

Create  test  package 

4 

4 

Create  content 

0 

0 

Prepare  content  prototype 

4 

0 

4 

Develop  and  obtain  approval  of  DM  requirements  list  for 
LDMs  and  develop  beta  training  products  in  runtime 
environment 

0 

3 

-3 

1 

Get  DMs  from  the  CSDB  via  the  API 

0 

1 

-1 

Design  interface  and  controls 

3 

3 

Design  for  accessibility 

2 

2 

Comply/conform  to  SCORM 

2 

0 

2 

1 

Review  ILE  content  prototype 

2 

2 

1 

Construct  assessments  (i.e.,  test  bank) 

3 

0 

3 

1 

Load  LDMs  into  CSDB 

0 

3 

-3 

Develop  test  design  (i.e.,  length,  question  type,  location, 
and  so  forth) 

4 

4 

Develop  storyboards,  trainees  guides,  and  lesson  plans 

0 

0 

Storyboards 

84 

68 

16 

1 

Trainee  guides 

16 

16 

Lesson  plans,  instructor  guides,  and  exercise 
controller  guides 

16 

16 

Portion-mark  content 

26 

26 

Review  storyboards,  trainees  guides,  and  lesson  plans 

3 

3 

Construct  assets  (i.e.,  media  and  so  forth) 

112 

112 

Construct  enabling  learning  objects 

22 

Construct  terminal  learning  objects 

3 

2 

1 

1 

Develop  work  flow;  review  and  approve  LDMs  and 
training  products 

0 

4 

-4 

Translate  LDMs  into  training  products  for  display  in  Web- 
based  browser 

0 

4 

-4 

Check  LDMs  in  CSDB 

0 

1 

-1 

Review  and  accept  learning  objects 

4 

2 

2 

1 

Publish  content  in  LMS 

11 

0 

11 

Publish  content  in  LMS  using  API 

0 

2 

-2 

Conduct  government  acceptance  testing 

4 

3 

1 
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Table  B-3.  Changes  in  Staff-Hours  To  Produce  Training  Courses  Using  the  Bridge  (Continued) 


Task 

Current 

Bridge 

Change 

Notes 

Submit  content  submission  form  on  content  forecasting 
system 

2 

2 

Deliver  course 

0 

0 

Execute  pre-pilot  checklist 

2 

2 

Perform  pre-pilot  conference 

2 

2 

Set  up  classroom 

1 

1 

Deliver  course  documents 

1 

1 

Conduct  Train-the-Trainer 

1 

1 

Identify  pilot  participants 

1 

1 

Perform  pilot 

2 

2 

Execute  post  pilot  checklist 

1 

1 

Finalize  training  course  control  document 

1 

1 

Perform  content  corrections 

8 

8 

Perform  post  pilot  conference 

1 

1 

Recommend  promulgation 

2 

2 

Promulgate 

1 

1 

Manage  content 

0 

0 

Review  ILE  content 

3 

3 

Register  SCORM-conformant  courseware  with  ADL-R 

1 

1 

Modify  technical  information  in  response  to  ECPs 

40 

1 

39 

Total 

426 

360 

66 

Note  1  for  Table  B-3:  All  Original  Equipment  Manufacturer  (OEM)  technical  data  are  stored  in  a  centralized  Common 
Source  Data  Base  (CSDB). 

Note  2  for  Table  B-3:  Constructing  learning  content  is  more  efficient. 

Note  3  for  Table  B-3:  Eliminates  the  time  to  collect  training  material  from  many  sources  and  locations. 
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Appendix  C 

Timeliness  of  Technical  Manuals 
and  Training  Courses 


Chapter  4  of  this  report  discusses  the  possibility  that  Navy  crews  might  not  have  the 
appropriate  Integrated  Logistics  Support  (ILS)  following  ship  alterations  (SHIPALTs). 
This  appendix  shows  that  the  Navy  has  had  a  policy  to  keep  this  from  happening  since 
2004.  The  following  paragraphs  cite  material  that  supports  this  policy  in  Naval  Sea  Sys¬ 
tems  Command  (NAVSEA)  Document  TS9090-310E,  Alterations  to  Ships  Accomplished 
by  Alteration  Installation  Teams.1  It  was  issued  April  2009  and  supersedes  TS9090- 
310D,  dated  February  2004. 

Section  1.5.1  (Naval  Supervising  Activity),  Item  15  states  that  the  Naval  Super¬ 
vising  Activity  (NSA)  has  the  responsibility  to  “ensure  that  Integrated  Logistics  Support 
(ILS)  products  from  the  AIT  [Alteration  Installation  Team]  are  properly  distributed.”  ILS 
products  include  technical  manuals.  The  NSA  and  the  Regional  Maintenance  and  Mod¬ 
ernization  Coordinating  Office  (RMMCO)  have  the  responsibility  to  ensure  that  the  ILS 
products  are  in  place  before  the  ship  leaves  the  yard. 

Appendix  H,  Definitions,  Item  33  states  that  the  NSA  is  “The  single  Naval  Activity 
charged  with  the  responsibility  of  oversight  of  work  being  accomplished  on  U.S.  Naval 
ships  during  any  type  of  availability,”  and,  moreover,  “NSAs  have  the  authority  and 
responsibility  to  preclude  and/or  stop  AITs  from  performing  work  when  they  are  found  to 
be  in  non-compliance  with  this  or  other  invoked  specifications.” 

However,  Section  1.5.4,  AIT  Manager,  allows  for  the  possibility  that  the  policy  may 
not  be  followed.  “In  cases  where  the  AIT  is  unable  to  complete  the  installation  within  the 
availability,  the  AIT  Manager  shall 

•  Document  the  amount  of  work  left  to  be  accomplished. 

•  Ensure  the  NSA  has  been  infonned. 

•  Verily  and  concur  in  writing  that  the  AIT  assessment  of  the  impact  of  the 
missing  equipment  or  capability  and  function  is  accurate  and  complete. 


1  Available  at  http://www.navsea.navy.mil/Organization/TS909Q-31QE  %20Apr09.pdf. 
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•  Add  amplifications  or  clarifications  as  appropriate. 

•  Obtain  NS  A  concurrence  that  all  required  mitigating  actions  and  documenta¬ 
tions  have  been  performed.” 

Appendix  C,  Messages  Checklists  &  Reports,  contains  a  fonn  called  Exceptions  to 
ILS  Verification.  This  form  lists  items  that  might  not  have  been  provided  upon  comple¬ 
tion  of  the  alteration  or  ship  change,  and  technical  manuals  are  one  such  item. 

NAVSEA  Document  TS9090-310E  clearly  indicates  that  the  Navy  does  allow  for 
the  possibility  that  the  appropriate  technical  manuals  and  training  courses  may  not  be  in 
place.  The  interesting  questions  are,  why  are  the  manuals/courses  not  ready  on  time  and 
what  are  the  incidence  of  delays? 
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Appendix  D 
Navy  Learning  Centers 


1 .  Center  for  Security  Forces  (CSF) 

2.  Center  for  Explosive  Ordnance  Disposal  (EOD)  and  Diving 

3.  Center  for  Naval  Engineering  (CNE) 

4.  Center  for  Naval  Leadership  (CNL) 

5.  Center  for  Seabees  and  Facilities  Engineering  (CSFE) 

6.  Submarine  Learning  Center  (SLC) 

7.  Center  for  Sea,  Air,  and  Land  (SEAL)  and  Special  Warfare  Combatant-craft 
Crewmen  (SWCC) 

8.  Center  for  Information  Dominance  (CENINFODOM) 

9.  Center  for  Force  Health  Protection/Naval  Medical  Education  and  Training  Com¬ 
mand  (NMETC) 

10.  Center  for  Naval  Aviation  Technical  Training  (CNATT) 

1 1 .  Center  for  Naval  Intelligence  (CENNAVINTEL) 

12.  Center  for  Personal  and  Professional  Development  (CPPD) 

13.  Center  for  Service  Support  (CSS) 

14.  Center  for  Surface  Combat  Systems  (CSCS) 
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Appendix  E 

Content  Hours  and  Computer-Based 
Training  (CBT)  Courses 
Delivered  by  Navy  e-Learning  (NeL) 


Table  E-1  shows  the  content  hours  and  courses  delivered  by  NeL,  a  part  of  the  Naval 
Education  and  Training  Command  (NETC),  from  2004  through  2009. 

Table  E-1.  Content  Hours  and  Courses  Delivered  by  NeL 


Fiscal  Year  (FY) 

2004 

2005 

2006 

2007 

2008 

2009 

Total  instructional  hours 

1,637 

2,196 

2,207 

4,124 

3,322 

3,292 

All  schools/centers  hours 

518 

1,564 

1,560 

1,353 

1,353 

1,353 

Courses 

Learning  centers 

49 

55 

34 

A  school  available 

7 

1 

6 

18 

288 

A  school  unavailable 

206 

517 

607 

155 

374 

C  school 

21 

58 

General  military  training 

16 

14 

14 

7 

11 

9 

Navy  department 

280 

186 

137 

1,346 

475 

532 

Total 

502 

724 

759 

1,584 

933 

921 

E-1 


Illustrations 


Figures 

Figure  3-1.  Current  Processes  for  Producing  Technical  Manuals  and  Training 

Courses . 3-2 

Figure  3-2.  Processes  for  Producing  Technical  Manuals  and  Training  Courses 

Using  the  Bridge . 3-3 

Tables 

Table  2-1.  Bridge  Project  Investment  Costs  (Full  Cost  to  the  Government) . 2-1 

Table  2-2.  Burdening  Allowances . 2-2 

Table  2-3.  Implementation  Costs  for  the  Aggregate  Sample . 2-2 

Table  2-4.  Hourly  Pay  Rates . 2-3 

Table  2-5.  Number  of  Course  Developers  Requiring  Bridge  Training . 2-4 

Table  2-6.  Production  of  CBT  Delivered  by  NeL . 2-4 

Table  2-7.  Professional  Skills  Involved  in  Producing  Training  Courses . 2-5 

Table  3-1.  Cost  Savings  for  Producing  the  Nominal  Products  Using  the  Bridge . 3-4 

Table  3-2.  Production  of  Navy  HM&E  Technical  Manuals  in  2009 . 3-6 

Table  3-3.  Cost  Savings  From  Producing  HM&E  Technical  Manuals  Using  the 

Bridge . 3-6 

Table  3-4.  Cost  Savings  From  Producing  NeL  Training  Courses . 3-6 

Table  3-5.  Cost  Savings  From  Producing  AN/AQS-20A  Technical  Manuals 

Using  the  Bridge . 3-7 

Table  3-6.  Cost  Savings  From  Producing  AN/AQS-20A  Training  Courses 

Using  the  Bridge . 3-8 

Table  4-1.  Bridge  Reductions  in  Delays  of  Delivering  Technical  Infonnation 

to  the  Fleet . 4-3 

Table  5-1.  Summary  of  Cost-Benefit  Analysis . 5-1 

Table  5-2.  Sensitivity  Analysis . 5-3 


F-l 


Abbreviations 


ADL-R 

Advanced  Distributed  Learning  Registry 

AIT 

Alteration  Installation  Team 

ALSP 

Acquisition  Logistics  Support  Plan 

AN/AQS-20A 

Towed  Underwater  Sonar  Mine  Detector 

API 

Application  Programming  Interface 

ATIS 

Advanced  Technical  Information  Support 

Bridge 

Collection  of  new  software  and  technical  and  business 
processes  used  for  the  integrated  management  and  production 
of  technical  data  and  learning  content  analyzed  in  this  study 

Bridge  Project 

The  RTOC  project  analyzed  in  this  report 

CBT 

Computer-Based  Training 

CDP 

Course  Design  Plan 

CENINFODOM 

Center  for  Information  Dominance 

CENNAVINTEL 

Center  for  Naval  Intelligence 

CIN 

Course  Identification  Number 

CNATT 

Center  for  Naval  Aviation  Technical  Training 

CNE 

Center  for  Naval  Engineering 

CNL 

Center  for  Naval  Leadership 

CPPD 

Center  for  Personal  and  Professional  Development 

CSCS 

Center  for  Surface  Combat  Systems 

CSDB 

Common  Source  Data  Base 

CSFE 

Center  for  Seabees  and  Facilities  Engineering 

CSF 

Center  for  Security  Forces 

CSS 

Center  for  Service  Support 

DM 

Data  Module 

DMC 

Data  Module  Code 

DMRL 

Data  Module  Requirements  List 

DVD 

Digital  Versatile  Disc  (formerly  Digital  Video  Disc) 

ECP 

Engineering  Change  Proposal 

EOD 

Explosive  Ordnance  Disposal 

FY 

Fiscal  Year 

G&A 

General  and  Administrative 

GS 

General  Schedule 

HM&E 

Hull,  Mechanical,  and  Electrical 

IDA 

Institute  for  Defense  Analyses 

IETM 

Interactive  Electronic  Technical  Manual 

ILE 

Integrated  Learning  Environment 

ILS 

Integrated  Logistics  Support 

LCS 

Littoral  Combat  Ship 

LDM 

Learning  Data  Module 

LMS 

Learning  Management  System 
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LSAR 

Logistic  Support  Analysis  Record 

NAVSEA 

Naval  Sea  Systems  Command 

NAVSSES 

Naval  Ship  Systems  Engineering  Station 

NeL 

Navy  eLearning 

NETC 

Naval  Education  and  Training  Command 

NIAPS 

Navy  Integrated  Application  Product  Suite 

NMCI 

Navy  Marine  Corps  Intranet 
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